Route a ticket, one call
10 scored items
- What it measures
- Can the model put a support ticket in the right one of 4 queues when asked directly, and answer with the queue's name only?
- How it is scored
- One point when the reply, trimmed and in lower case, is exactly the queue in the answer key. 10 tickets, one call each.
- Where points are lost
- A model loses the point by being wrong, or by being right in the wrong form: a sentence, a queue in quotes or in bold. A strict reader counts both the same, as a program would.
- How to do better
- Ask for one word and name the words allowed. Better still, use a typed output, as in the third test, so the form cannot drift.
- Test set and key:
- scripts/ch2_course_collapse.py