Smart DVI

The app

Smart DVI is a vehicle inspection app for the Meta Quest 3, and auto shops use it in production. The technician wears the headset while working through the shop’s inspection checklist. They grade each item, take photos by looking at the part, and record notes by voice. The finished inspection goes to the service advisor as a repair order, and the customer approves or declines it line by line.

I’m the lead engineer. I run the app’s development, and I own its AI, which covers photo grading, the agents on the headset, VIN reading and speech. I also write the specs the backend and web teams build from, citing the code behind each defect.

What the AI does

The readings run as agents on the headset. An agent runs when the technician takes a photo, records a note or connects to the car. Shops turn agents on per checklist item and choose whether each one fills in the answer or only suggests it. An agent never changes a value the technician entered, and it only suggests a reading it isn’t confident about.

VIN reading tries cheap methods before a model call. The headset takes a burst of frames and keeps the sharpest. It tries to decode the barcode on the full frame, then on a cropped, grayscale, contrast-boosted copy. Only if both fail does a vision model read the plate, and it has to return exactly seventeen characters, marking any it isn’t sure of. The VIN’s check digit then corrects those.

Photo grading

Each photo gets one structured reply from a vision model. The reply holds the suggested grade and a separate verdict on whether the photo shows the item at all, so the relevance check costs no extra call and can’t contradict the grade.

To choose the model and prompt, I built a test set from real inspection photos with two answer keys. One is the technician’s own grade. The other is a blind grade, written against a fixed standard by a grader who never saw the technician’s grade, notes or voice. I changed one thing at a time and scored each version against both keys, and the backend adopted the version that came out ahead.

The same tests turned up two things that shaped the product. The model’s stated confidence doesn’t predict when its grade is wrong, so technicians confirm every grade. And when I added false context to the request, such as an old note that contradicted the photo, accuracy fell sharply.

I also cut the wait between taking a photo and seeing its grade roughly in half. Sending smaller images removed a large share of the input with no loss in accuracy, and the reply now streams with the grade first.

Getting findings to the customer

I went through production inspection data to find where technicians’ findings were getting lost before they reached the repair order. Two fixes came out of it. An item with a photo, clip, voice note or reading can no longer be marked complete without a grade, and the row tells the technician why. The app also checks at the car whether a photo actually shows the item.

I run the pilot in person. I sat with the technician using the app, wrote down what got in the way, traced each problem to its cause in the code and shipped the fixes.

The headset app

The whole inspection works offline and syncs when the headset reconnects. I built the app from reusable pieces, so features like Bluetooth diagnostics, voice and the agent framework can move to other products.

For headsets already in shops, I built remote diagnostics, so support can find a problem without touching the headset or shipping a new build.

I built the dataset that turns the car’s trouble codes into names a technician can read. It cross-checks public code lists that didn’t copy one another and keeps the lineage of every title. It covers thousands of codes that used to show up bare.

I also merged several separately built versions of the app into one. The app’s logic runs in test harnesses without the game engine, so logic changes can be checked without a headset.