Data Cleaning in Data Mining
Quality of your data is critical in getting to
final analysis.Any data which tend to be incomplete, noisy and inconsistent can
effect your result.
Data cleaning in data mining is the process of
detecting and removing corrupt or inaccurate records from a record set, table
or database.
Some data cleaning methods :-
1 You can ignore the tuple.This is done when
class label is missing.This method is not very effective , unless the tuple
contains several attributes with missing values.
2 You can fill in the missing value
manually.This approach is effective on small data set with some missing values.
3 You can replace all missing attribute values
with global constant, such as a label like “Unknown” or minus infinity.
4 You can use the attribute mean to fill in
the missing value.For example customer average income is 25000 then you can use
this value to replace missing value for income.
5 Use the most probable value to fill in the
missing value.
Noisy Data
Noise is a random error or variance in a
measured variable. Noisy Data may be due to faulty data collection instruments,
data entry problems and technology limitation.
How to Handle Noisy Data?
Binning:
Binning methods sorted data value by
consulting its “neighbor- hood,” that is, the values around it.The sorted
values are distributed into a number of “buckets,” or bins.
For example
Price = 4, 8, 15, 21, 21, 24, 25, 28, 34
Partition into (equal-frequency) bins:
Bin a: 4, 8, 15
Bin b: 21, 21, 24
Bin c: 25, 28, 34
In this example, the data for price are first
sorted and then partitioned into equal-frequency bins of size 3.
Smoothing by bin means:
Bin a: 9, 9, 9
Bin b: 22, 22, 22
Bin c: 29, 29, 29
In smoothing by bin means, each value in a bin
is replaced by the mean value of the bin.
Smoothing by bin boundaries:
Bin a: 4, 4, 15
Bin b: 21, 21, 24
Bin c: 25, 25, 34
In smoothing by bin boundaries, each bin value
is replaced by the closest boundary value.
Regression
Data can be smoothed by fitting the data into
a regression functions.
Clustering:
Outliers may be detected by clustering,where
similar values are organized into groups, or “clusters.Values that fall outside
of the set of clusters may be considered outliers.

Clustering
Super consept peer basha sir title super fazil
ReplyDeleteGood..ππ
ReplyDeleteNice
ReplyDeleteSuper
ReplyDeleteExcellent
ReplyDeleteExcellent
ReplyDeleteGood bro
ReplyDeleteSuch as explanation is gud
ReplyDeletefaizal
Excellent bro
ReplyDeleteNice one
ReplyDeleteWoohoo you did it bro haaa quite impressive ..
ReplyDeleteNice post
ReplyDeleteNice explanation
ReplyDeletesuper
ReplyDeleteNice
ReplyDeleteConcept is unique
ReplyDeleteGud explanation fazil
ReplyDeleteYour presentation is always extraordinary
ReplyDeleteNice da
ReplyDeleteSuper bro
ReplyDeleteNice explanation and content is good
ReplyDeleteGud explanation fazil
ReplyDeleteGud data with unique standard. .
ReplyDeleteGo aheadπ
Good work,keep going man..
ReplyDeleteFazil this very very amazing explanation, good work
ReplyDeleteHard work never fails...
This comment has been removed by the author.
ReplyDeleteGood work bro keep doing continuously,make effort to do more like this.
ReplyDeleteNice job
ReplyDeleteExcellent bro
ReplyDeleteNice concept & super
ReplyDeleteNice concept & super
ReplyDeleteCongratulation you got 5 Marks
ReplyDelete