Ride Sharing Driver Sufficiency
2023 — A model that predicts whether an area will be short of drivers in a given hour
01 Problem
When a rider asks for a car and no driver is free nearby, the request fails, and a rider who is turned away often enough stops opening the app. Scotty, a ride sharing and delivery service in Istanbul, wants to know ahead of time where and when that will happen, so it can move drivers or offer them incentives before the demand arrives.
The goal is a classifier that takes a pick-up area and an hour, and predicts whether Scotty will have enough drivers there.
02 Cleaning
Each row of the raw data is one trip request from October to December 2017, with the pick-up area, the start time, and a status such as confirmed or nodrivers. 14,901 requests have no trip ID and 14,900 have no driver ID. These are not gaps to fill in. They are the requests that never found a driver, and the status column already says so.
Only the start time, the area, and the status are kept, and every start time is rounded down to the hour.
train <- read.csv("data_input/data-train.csv")
train %>% is.na() %>% colSums()
train <- train %>% select(start_time, src_area, status)
train <- train %>%
mutate(start_time = floor_date(ymd_hms(start_time), unit = "hour")) %>%
mutate_if(is.character, as.factor) %>%
rename(datetime = start_time) %>%
arrange(datetime)03 Target
The model predicts hours, not single requests. The requests that found no driver are counted per area and hour, and the data is padded so every area has a row for every hour, with a count of zero where nothing failed.
An hour is insufficient if at least one request in it found no driver, and sufficient otherwise.
train <- train %>%
group_by(src_area, datetime, status) %>%
summarise(count = n()) %>%
arrange(src_area, datetime)
train <- train %>%
filter(status == "nodrivers") %>%
select(-status)
train <- train %>%
group_by(src_area) %>%
summarise(datetime = seq(min(datetime), max(datetime), by = "hour")) %>%
pad() %>%
left_join(train, by = c("src_area", "datetime")) %>%
arrange(src_area, datetime) %>%
mutate(count = ifelse(is.na(count), 0, count))
train <- train %>%
mutate(coverage = ifelse(count > 0, "insufficient", "sufficient") %>% as.factor()) %>%
select(-count) %>%
as.data.frame()04 Features
The hidden test set only gives a pick-up area and a datetime, so the model can only use what those two carry, which are the area (one of sxk3, sxk8, and sxk9), the hour of day, and the weekday.
The hour is a category rather than a number, so the model can learn a different answer for every hour instead of one straight trend through the day.
train <- train %>%
mutate(dhour = as.factor(hour(datetime)),
wday = as.factor(wday(datetime, label = T))) %>%
select(-datetime) %>%
as.data.frame()05 Split
The area-hours are split 80/20 at random. The 904 validation hours hold 493 insufficient and 411 sufficient, close enough to balanced that accuracy means something.
Two areas are far from balanced on their own, with sxk8 almost always covered and sxk9 almost always short. Upsampling the rarer class inside each area was tried, and the models scored worse, so the data is used as it is.
RNGkind(sample.kind = "Rounding")
set.seed(1)
indices <- sample(nrow(train), nrow(train) * 0.8)
train_data <- train[indices, ]
test_data <- train[-indices, ]
X_train <- train_data %>% select(-coverage)
y_train <- train_data$coverage
X_test <- test_data %>% select(-coverage)
y_test <- test_data$coverage06 Models
Insufficient is the positive class. A missed shortage leaves riders without a car, and a false alarm sends drivers where nobody needs them. Both cost Scotty, so accuracy is the main metric, with sensitivity and specificity kept in balance.
Three classifiers are trained. They are naive Bayes with Laplace smoothing, a conditional inference tree loosened so it splits more easily, and a random forest tuned with 5-fold cross validation repeated 7 times.
On validation they are nearly tied. Naive Bayes reaches 78.3% accuracy, the decision tree reaches 78.6%, and the random forest reaches 78.9%. That gap is too small to pick a winner.
model_nb <- naiveBayes(x = X_train,
y = y_train,
laplace = 1)
model_dt <- ctree(coverage ~ .,
train_data,
control = ctree_control(mincriterion = 0.35,
minsplit = 5,
minbucket = 3))
ctrl <- trainControl(method = "repeatedcv",
number = 5,
repeats = 7)
model_rf <- train(x = X_train,
y = y_train,
trControl = ctrl)07 Evaluation
On validation the decision tree catches 426 of the 493 hours that were short of drivers, a sensitivity of 86%, and raises a false alarm on 126 of the 411 covered hours.
Its AUC of 0.83 means that, given a random short hour and a random covered one, it ranks the short one as riskier 83% of the time.
pred_dt <- predict(model_dt, X_test)
confusionMatrix(pred_dt, y_test)
pred_dt_raw <- predict(model_dt, X_test, type = "prob")
plotROC(pred_dt_raw[, 1], ifelse(y_test == "insufficient", 1, 0))08 Hidden week
Algoritma, the data science school this capstone was for, scores submissions on a hidden week of 504 area-hours, which is every hour from December 3 to December 9, 2017 in each of the 3 areas. Its labels are never shown, so it is the honest test.
All three models were submitted, and the decision tree scored best, with 82% accuracy, 89% recall, 81% precision, and 74% specificity. Validation could not separate the three, but the hidden week did.
test <- test_raw %>%
select(-coverage) %>%
mutate(src_area = as.factor(src_area),
datetime = ymd_hms(datetime),
wday = as.factor(wday(datetime, label = T)),
dhour = as.factor(hour(datetime)))
test$coverage_dt <- predict(model_dt, test)
submission <- test_raw %>% mutate(coverage = test$coverage_dt)
write.csv(submission, "submission-ardian-dt.csv")09 Location
In the hidden week the tree predicts sxk9 short of drivers in 168 of its 168 hours, and sxk8 in only 1. For those two areas, the location alone decides.
sxk3 is the mixed one, short in 131 of its 168 hours.
10 Hours
In sxk3 the hour decides. Every night from 02:00 through 06:00 the tree expects enough drivers, and almost every other hour it expects a shortage. The one hour that changes with the day is 22:00, short on 5 of 7 days.
For Scotty that reads as a schedule, with more drivers in sxk9 all day, more in sxk3 outside the early morning, and sxk8 left as it is.
11 Interpretation
LIME explains a single prediction by fitting a simple model around it. On the first four hours of the hidden week, all in sxk3 on a Sunday, the tree gives 00:00 and 01:00 a 62% chance of a shortage, and 02:00 and 03:00 only 29%. In the first three cases the area pushes toward a shortage, while the hours 02:00 and 03:00 push hard toward enough drivers.
The explanation fits are only 0.037 to 0.127, on a scale where 1 means the simple model matches the tree perfectly around that case. So these are rough local hints, not proof. The tree's own predictions per area and hour tell the story more reliably.
model_type.party <- function(x){
return("classification")
}
set.seed(1)
explainer <- lime(x = X_train, model = model_dt)
explanation <- explain(test %>% select(src_area, dhour, wday) %>% slice(1:4),
labels = "insufficient",
explainer = explainer,
n_features = 3)
explanation %>% plot_features()Ride Sharing Driver Sufficiency
82% accurate on a hidden week — catches 89% of driver shortages