Statistics in Data Science

-

 What is Statistics?

- Study of the collection, analysis, interpretation, presentation and organization of Data

- Problem Statement -> Data Analysis -> Informed Business Decisions (Problems Solved)

- Mean, Median, Mode etc. are Statistical formulas

- Statistical Principles:

Data should be normally distributed

Linear Regression: Relation between variables should be linear

- Categories of Statistics

- Descriptive Analytics

- used when we have full data for given population

- Inferential Analytics

- used when there is incomplete data for given population (eg. exit polls)

- used when it is not feasible to examine every member/analyze entire population data

- we study a random sample and describe/make inferences about population

- Statistical Analysis Considerations

-purpose is clear and well-defined

-document questions in advance

-define population of interest (based on purpose)

-determine sample (based on purpose of study)

-sample must be random to represent characteristics of population


Comments

Post a Comment

Popular posts from this blog

MovieLens Case Study with Python

-
Image
Problem Objective : Here, we ask you to perform the analysis using the Exploratory Data Analysis technique. You need to find features affecting the ratings of any particular movie and build a model to predict the movie ratings. Domain: Entertainment Dataset Description : These files contain 1,000,209 anonymous ratings of approximately 3,900 movies made by 6,040 MovieLens users who joined MovieLens in 2000. Analysis Tasks to be performed: 1. Import the three datasets In [1]: import numpy as np #import the numpy library for numerical operations import pandas as pd #import the pandas library for data wrangling operations In [2]: colnames_movies = [ 'MovieID' , 'Title' , 'Genres' ] df_movies = pd . read_csv ( 'movies.dat' , delimiter = '::' , engine = 'python' , names = colnames_movies ) #open the movies.dat datafile df_movies . head () Out[2]: MovieID Title Genres 0 1 Toy Story (1995) Animation|Children's|Comedy 1 2
Read more

Data Pre-processing for Machine Learning

-
Image
  Traditional Computer Computation: ¶ (Data & Program) -> Computer -> Output Machine Learning: ¶ (Data & Output) -> Computer -> Program/Model The model that is generated is used to predict the results for the future Machine Learning (ML) is subset of Artificial Intelligence (AI) In this exercise, let us learn below Data Pre-processing Techniques: Data Acquisition Data Exploration Data Wrangling Data Manipulation In [3]: import os os . getcwd () #gets current working directory Out[3]: 'C:\\Users\\ns45237\\working_neeraj\\2_machine_learning' In [38]: #Data Acquisition #Loading the CSV file in Python import pandas as pd #pandas is python data wrangling package df = pd . read_csv ( 'BostonHousing.csv' ) #read the CSV file into a Python DataFrame #df.to_csv("/home/neerajshinde/Data/BostonHousing.csv") #loading data into an existing CSV file #df = pd.read_excel('BostonHousing.xlsx') #read the XLSX file into a Python
Read more