House Price Prediction

What You'll Build

By the end of this project you'll have a complete tabular ML pipeline that:

• Loads and explores the Ames Housing dataset (1,460 homes, 79 features)
• Performs exploratory data analysis (EDA) to find patterns and outliers
• Engineers new features and handles missing values properly
• Trains three models and compares them with cross-validation
• Tunes the best model and reports a final RMSE on a held-out test set

Project Setup

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

# Install dependencies
pip install pandas numpy matplotlib seaborn scikit-learn xgboost jupyter

Create a house_price.ipynb notebook and follow along section by section.

Step 1 — Load and Inspect the Data

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

train = pd.read_csv('train.csv')
test  = pd.read_csv('test.csv')

print(train.shape)          # (1460, 81)
print(train.dtypes.value_counts())
print(train['SalePrice'].describe())

Key observations from the first look:

• Target: SalePrice — continuous, right-skewed (we'll log-transform it)
• Features: 36 numeric + 43 categorical
• Missing values: 19 columns have nulls; some (like PoolQC) mean "no pool", not truly missing

Step 2 — Exploratory Data Analysis

Target Distribution

fig, axes = plt.subplots(1, 2, figsize=(12, 4))
train['SalePrice'].hist(bins=50, ax=axes[0])
axes[0].set_title('SalePrice (raw)')

np.log1p(train['SalePrice']).hist(bins=50, ax=axes[1])
axes[1].set_title('SalePrice (log-transformed)')
plt.tight_layout()
plt.show()

Correlation Heatmap (Top Features)

numeric = train.select_dtypes(include=np.number)
corr = numeric.corr()['SalePrice'].abs().sort_values(ascending=False)
print(corr.head(15))

The top correlated numeric features are: OverallQual, GrLivArea, GarageCars, GarageArea, and TotalBsmtSF.

top_feats = corr.index[1:11]  # exclude SalePrice itself
sns.heatmap(
    numeric[list(top_feats) + ['SalePrice']].corr(),
    annot=True, fmt='.2f', cmap='coolwarm'
)
plt.title('Top 10 Feature Correlations with SalePrice')
plt.show()

Outliers

plt.scatter(train['GrLivArea'], train['SalePrice'], alpha=0.4)
plt.xlabel('GrLivArea'); plt.ylabel('SalePrice')
plt.title('Living Area vs Sale Price')
plt.show()

# Two large houses sold cheap — likely non-market sales, remove them
train = train[~((train['GrLivArea'] > 4000) & (train['SalePrice'] < 300_000))]

Step 3 — Feature Engineering

Handle Missing Values

def fill_missing(df):
    # "NA" in docs means the feature doesn't exist — fill with "None"
    none_cols = ['PoolQC','MiscFeature','Alley','Fence','FireplaceQu',
                 'GarageType','GarageFinish','GarageQual','GarageCond',
                 'BsmtQual','BsmtCond','BsmtExposure','BsmtFinType1','BsmtFinType2',
                 'MasVnrType']
    for c in none_cols:
        if c in df.columns:
            df[c] = df[c].fillna('None')

    # Numeric NAs where 0 makes sense
    zero_cols = ['GarageYrBlt','GarageArea','GarageCars',
                 'BsmtFinSF1','BsmtFinSF2','BsmtUnfSF','TotalBsmtSF',
                 'BsmtFullBath','BsmtHalfBath','MasVnrArea']
    for c in zero_cols:
        if c in df.columns:
            df[c] = df[c].fillna(0)

    # Fill remaining with mode or median
    for c in df.columns:
        if df[c].isnull().any():
            if df[c].dtype == 'object':
                df[c] = df[c].fillna(df[c].mode()[0])
            else:
                df[c] = df[c].fillna(df[c].median())
    return df

train = fill_missing(train)
test  = fill_missing(test)

Create New Features

for df in [train, test]:
    df['TotalSF']        = df['TotalBsmtSF'] + df['1stFlrSF'] + df['2ndFlrSF']
    df['TotalBathrooms'] = (df['FullBath'] + 0.5 * df['HalfBath']
                            + df['BsmtFullBath'] + 0.5 * df['BsmtHalfBath'])
    df['HouseAge']       = df['YrSold'] - df['YearBuilt']
    df['RemodAge']       = df['YrSold'] - df['YearRemodAdd']
    df['IsRemodeled']    = (df['YearBuilt'] != df['YearRemodAdd']).astype(int)

Encode Categoricals

combined = pd.concat([train.drop('SalePrice', axis=1), test], axis=0)
combined = pd.get_dummies(combined)

X_train = combined.iloc[:len(train)]
X_test  = combined.iloc[len(train):]
y_train = np.log1p(train['SalePrice'])

Step 4 — Model Comparison with Cross-Validation

from sklearn.linear_model import Ridge
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import cross_val_score
from xgboost import XGBRegressor

def rmse_cv(model, X, y, cv=5):
    scores = cross_val_score(
        model, X, y,
        scoring='neg_root_mean_squared_error',
        cv=cv
    )
    return -scores.mean(), scores.std()

models = {
    'Ridge':         Ridge(alpha=10),
    'Random Forest': RandomForestRegressor(n_estimators=200, random_state=42),
    'XGBoost':       XGBRegressor(n_estimators=500, learning_rate=0.05,
                                  max_depth=4, random_state=42, n_jobs=-1),
}

for name, model in models.items():
    mean, std = rmse_cv(model, X_train, y_train)
    print(f'{name:20s}  RMSE: {mean:.4f} ± {std:.4f}')

Expected results (approximate):

Model	CV RMSE (log scale)
Ridge	~0.135
Random Forest	~0.143
XGBoost	~0.127

Step 5 — Hyperparameter Tuning (XGBoost)

from sklearn.model_selection import GridSearchCV

param_grid = {
    'n_estimators': [400, 600],
    'max_depth':    [3, 4, 5],
    'learning_rate':[0.03, 0.05],
    'subsample':    [0.8, 1.0],
}

xgb = XGBRegressor(random_state=42, n_jobs=-1)
gs  = GridSearchCV(xgb, param_grid, cv=5,
                   scoring='neg_root_mean_squared_error',
                   verbose=1)
gs.fit(X_train, y_train)

print('Best params:', gs.best_params_)
print('Best RMSE:  ', -gs.best_score_)

Step 6 — Final Predictions

best_model = gs.best_estimator_
best_model.fit(X_train, y_train)

y_pred_log = best_model.predict(X_test)
y_pred     = np.expm1(y_pred_log)   # reverse the log1p transform

submission = pd.DataFrame({'Id': test['Id'], 'SalePrice': y_pred})
submission.to_csv('submission.csv', index=False)
print(submission.head())

Step 7 — Feature Importance

import matplotlib.pyplot as plt

feat_imp = pd.Series(
    best_model.feature_importances_,
    index=X_train.columns
).sort_values(ascending=False).head(20)

feat_imp.plot(kind='barh', figsize=(8, 8))
plt.title('Top 20 Feature Importances — XGBoost')
plt.gca().invert_yaxis()
plt.tight_layout()
plt.show()

You should see OverallQual, TotalSF, and GrLivArea dominating — consistent with the correlation analysis from EDA.

What to Try Next

• Stacking: Average predictions from Ridge + XGBoost + LightGBM for a small but consistent boost
• More feature engineering: Neighborhood price averages, basement/garage interaction terms
• Ordinal encoding: Replace pd.get_dummies for quality columns with ordered numeric values (Poor=1, Fair=2, … Excellent=5)
• Deploy: Wrap the model in a Flask or FastAPI endpoint and build a simple input form