Evaluation

What It Is

Determine whether a system meets expectations using explicit criteria and evaluation methods.
How AI evaluation differs from conventional software testing
- Traditional software testing requires additional technologies in the face of open output, probabilistic behaviour and the difficulty of directly determining correctness.
  - 1. Move from binary test assertions to graded quality judgments
  - 1. Validate the evaluator itself, not just the system under test
  - 1. Evaluate statistical performance rather than relying on one test result
- AI evaluation extends traditional testing by introducing evaluators capable of judging the quality of open-ended outputs.
  - The scorer type / Rules scorer Traditional assertion, procedural check JSON Schema / Function scorer, perform tests and check results. / Similarity scorer Compares RUGE, semantic similarity / AI Scorer LLM-as-a-Judge Evaluation answer accuracy, readability / Manual rating Manual evaluation of professional content
Key Idea
- Evaluation-Driven Development

Why AI Evaluation Is Challenging

Quality may not be directly quantifiable.
Open-Ended Tasks
- Many tasks have no single correct answer.
As systems become more capable, evaluating them also becomes harder.
Model behavior is often difficult to inspect, especially for closed models.
Benchmarks can lag behind model capabilities.

How to Evaluate

1. Define Evaluation Criteria
- What dimensions should be evaluated?
  - 1. Domain-Specific Capability: Can it perform the task?
    - What It Is
      - Whether the model can perform tasks in a given domain
    - What to Evaluate
      - Coding Ability
      - Mathematics
      - Scientific knowledge
      - Reasoning Ability
      - Domain Knowledge
  - 1. Generation Quality: How well does it perform?
    - What It Is
      - Quality requirements for model-generated content
    - What to Evaluate
      - Basic quality
        
        Fluency: Whether expression is natural
        
        Coherence: legitimacy of structure and logic
        
        Faithfulness: loyalty to original content
        
        Relevance: coverage of critical information
      - Factual Consistency
        
        What It Is
        
        Is there a factual basis for generating content
        
        Local factual consistency
        
        Whether to match given context
        
        For RAG, Text Abstract
        
        Global factual consistency
        
        Whether external credible facts are met
        
        Applicable to generic questions and answers, fact-finding
      - Safety and Security
        
        Harmful content or recommendations
        
        Discrimination and hate speech
        
        Content of violence
  - 1. Command compliance capability: Are the requirements met?
    - What It Is
      - Is the model capable of fulfilling its assigned requirements?
    - What to Evaluate
      - Format constraints
        
        Structured output of JSON, YAML etc.
      - Length Limit
        
        Restrictions on the number of words, paragraphs, etc.
      - Keyword constraints
        
        Must contain or prohibit the inclusion of specified words
      - Content constraints
        
        Discussion of specified themes or ranges only
      - Language and style constraints
        
        Specify language, tone, expression style
      - Role playing
        
        Maintaining role style and knowledge consistency
  - 1. Cost and delay: is this a cost-effective exercise?
    - What It Is
      - Whether model speed and cost are realistic
    - What to Evaluate
      - Latency
        
        First Token Delay (TTFT)
        
        Every Token Generation Time (TPOT)
        
        Total Response Time
      - Cost
        
        Input, Output Token Costs
        
        Resource costing from hosting
        
        Engineering and maintenance costs
1. Choose Evaluation Methods
- Available Methods
  - Language model indicators
    - How good is the model predicting the next Token?
      - Subtopic 2
        
        Neptunium
        
        What It Is
        
        The average amount of information a token carries.
        
        Units
        
        bit
        
        Role
        
        The bigger the entropy, the bigger the token, the bigger the special.
        
        The bigger the entropy, the harder to predict the next token.
        
        Measuring the unpredictability of the data itself
        
        Crossing entropy
        
        What It Is
        
        Measuring the difficulty of LM predicting the next token in the data set
        
        Units
        
        bit
        
        Role
        
        Measuring the performance of a model predicting these data
        
        BPC and BPB
        
        BPC
        
        What It Is
        
        Model predicts the average bit cost per character
        
        If each token takes 6bit and each token can carry 2 characters, each token takes 6/2 = 3
        
        Units
        
        bits/character
        
        BPB
        
        What It Is
        
        Average projected bit cost per byte
        
        If 8bit per character, then 3/(7/8)
        
        Units
        
        bits/byte
        
        Why It Is Needed
        
        Token splits differ from language model to language. / For example: / Model A takes hello as a Token. / Model B breaks it into five characters, Token. / It’s not fair to compare directly with each Token’s crossbow. / There is therefore a need to convert the forecast costs into the same units.
        
        Confusion
        
        What It Is
        
        Index forms of entropy or cross-permium
        
        It’s essentially like entropy.
        
        Role
        
        Measuring uncertainty in predicting the next token
        
        Regular
        
        The higher the structure, the lower the confusion.
        
        The larger the vocabulary, the greater the confusion.
        
        The longer the context, the lower the confusion.
      - What is it:
        
        The probabilities of a model assigned to real Token measure its projected performance.
  - Accurate assessment
    - Right? The code runs, right?
      - What It Is
        
        The output is automatically rated using clear inspection rules or reference answers.
        
        Example
        
        Other Organiser / Model Generation Code / ↓ / Run Cell Test / ↓ / Adopted: Correct / Existence failure: not entirely correct
      - Subtopic 3
        
        Functional correctness
        
        What is it: to determine whether the AI output fulfils the functional requirements by carrying out tests or checking binding conditions.
        
        For example, AI generates a sorting algorithm code, and we judge it by unit testing.
        
        Did the mission go right?
        
        Comparison with reference answers
        
        What is it? Calculating the similarities between AI output and pre-prepared reference answers.
        
        How much is the result like the standard answer?
  - AI as judge
    - How’s the code? Is the explanation reasonable?
      - Why is it not possible to use accurate assessments in their entirety?
        
        Because some of the questions are not clearly correct.
      - What It Is
        
        Use another AI model to evaluate the output of the target model according to specified criteria.
        
        Example
        
        Other Organiser / Enter / Code / Assessment criteria / Readability / Maintenanceability / Whether there is a clear concurrent risk / AI Decision / ↓ / Output rating and basis
      - Notes
        
        Consistency: The same type of input has been evaluated many times, and are similar results obtained?
        
        Accuracy: does the judgement’s rating really meet the expectations?
        
        Differing: Do you prefer your own answers, front-line answers or longer answers?
  - Comparative assessment
    - Who’s better - Model A and Model B, who’s better?
      - Why is a comparative assessment needed?
        
        Because sometimes it is easier to judge which is better than to give an absolute score to each result. / For example: / I’ll give you a score of two technical articles. You probably don’t know if it’s 82 or 85. / But if you put the two articles together, it might be easy for you to judge which one is clearer. / This is the value of a comparative assessment.
      - What It Is
        
        Directly compares the output of multiple models, and rankings are based on the comparison results.
        
        Example
        
        Same Job / ↓ / Model A generation code / Model B Generation Code / ↓ / Compare two codes. / ↓ / A better / B better / draw / ↓ / Comparison of multiple tasks / ↓ / Calculating Winning or Model Ranking
- How to Implement It
  - Precedence of precision; consideration of AI decisions when they cannot be accurately validated; review of significant findings in conjunction with manual or independent evidence.
    - The first question: Can the results be verified directly?
      - Yes, priority is given to functional correctness tests or procedural inspections.
      - For example, GoTiny is correctly implementing tools, returns JSON is compatible with Schema.
    - Second question: Is there a reliable reference answer or evidence?
      - Yes, precise matching, reference to similarity of answer, NLI or other means of evidentiary verification are considered.
      - Note that similarity of reference answers only provides a similar signal and does not directly justify the result.
    - Third question: Does it require subjective judgment?
      - If readability, consistency, style of expression, etc. need to be evaluated, the AI decision or manual assessment is considered.
    - Fourth question: Do you want to compare multiple models?
      - If needed, a comparative assessment could be made on the basis of the above methodology.
1. Design and Implement the Evaluation Pipeline
- What It Is
  - Organize assessment criteria, methodologies and data into a repertoireable assessment process
- Why
  - Need for objective measurement of system quality
  - Need to identify and locate failure links
  - Need to verify improvements to different versions
- How to Do It
  - 1. Targeting assessments
    - Why
      - Only the final results are evaluated and it is difficult to locate intermediate steps
      - It’s like a clear test target.
    - How to Do It
      - Selection of specific criteria and methods
      - Level of assessment established
        
        Task Level (Task)
        
        What It Is
        
        Evaluate whether the entire user mandate is fulfilled
        
        For Example
        
        Agent successfully repaired the code defect
        
        Rotation secondary (Turn)
        
        What It Is
        
        Evaluation of the quality of each round of interactions
        
        For Example
        
        Agent, is this round correctly understanding user requirements?
        
        Intermediate Output Level
        
        What It Is
        
        Evaluation of the output of key components within the system
        
        For Example
        
        Whether tool selection is correct
        
        Whether the search results are relevant
        
        Whether the tool execution results are correctly understood
        
        End-to-end tests, interactive process checks and component tests can be adapted to traditional software projects.
      - Rabric
        
        How to establish a rating criteria with examples.
        
        For example, evaluating the readability of technical articles: / - One: Logical confusion, difficult to understand. / - Three minutes: basic clarity, but partly inadequate conceptual interpretation. / - 5 minutes: Clear structure, precise concepts, full examples. / This is directly related to the general test assertion. / Traditional tests are usually: / assert(actual == expected) / The AI Open Task does not necessarily have the only expected result, so there is a need to translate what is good into an enforceable scoring rule.
  - Preparation of assessment data
    - What It Is
      - Prepare representative test samples that can be used to validate the quality of the system
      - It’s like a test case.
    - Data Type
      - Normal scene
        
        Overwrite common tasks
      - Border scene
        
        Long input, special format, etc.
      - History Failed
        
        Verify detected problems
      - Cross-border scene
        
        Whether or not the certification system correctly rejects or processes a request beyond its scope
  - 1. Operation of systems and implementation evaluation
    - What It Is
      - Evaluation of models or systems to be evaluated using the methodology selected in step 2
      - Similar to the test application
    - How to Evaluate
      - Development phase: Focus on off-line assessment.
        
        It’s like automated and regression tests.
      - Production phase: continuous observation using online assessment.
        
        Similar to observable. Blogs, metrics and Traces
  - Analysis of findings
    - What It Is
      - Translating test results into a system quality judgement
    - How to Do It
      - Statistical corporate indicators
      - Analyse failure cases
      - Conducting data slices on demand
        
        Results by test case category
        
        Discovery of problems covered by overall indicators
        
        Data Slicing
        
        What It Is
        
        Split data into subsets according to different characteristics
        
        Why
        
        Overall average indicators may mask local problems
        
        Suppose you’ve prepared 100 test examples to verify that Agent was able to select the tool and complete the task correctly. / Your assessment is that the mission success rate is at least 85 per cent. / Implementation result: / Test cases Number of success Rate / All tests 100 86 86% / Result: 86 per cent over 85 per cent and the overall assessment was adopted. / But now, we’re sorting 100 test cases by input length. / Data slices. Number of cases. Success. / Short input tasks 80 76 95% / Long input task 20 10 50% / Total 100 86 86% / What did you find at this time? / The overall success rate of the model is 86 per cent, but it is only 50 per cent for long-input tasks. / In other words, the overall performance looks good, but in fact there is a clear weakness. / If we look only at the overall indicators, this problem may be overlooked. / That’s what data slices mean.
        
        How to Do It
        
        Split by task type
        
        Cut by input length
        
        Split by data source
        
        Evaluate each subset separately
        
        Notes
        
        Simpson’s paradox.
        
        Model A performed better as a whole than Model B, but Model B performed better than Model B in each corresponding subset.
        
        For example: / Task Type Model A Model B / Simple tasks 90/100 = 90% 19/20 = 95% / 1/10 = 10% 12/20 = 60% / Total 91/110 = 82.7% 31/40 = 77.5% / Observations: / - Simple task: B better. / - Hard task: B better. / - Overall success rate: A is higher. / Why? / Because Model A tests are concentrated, simple tasks are higher. / Thus, in comparing the model, not only the overall success rate, but also the sample distribution of the different test sets. / This is also why, when assessing the two models, the same test assignments should normally be used as much as possible. Otherwise, the overall score could be misleading. / However, the Simpson paradox is not a complex issue that needs to be analysed individually every time a data slice is made, and you can understand why it was created.
        
        Similar to the test case classification
      - Compare different versions of performance
  - 1. Decision-making based on assessment findings
    - What It Is
      - Use the results of the assessment to determine whether the system meets the needs and decide whether to select, optimize or continue to validate
    - Why
      - Evaluation is not an end in itself.
      - Assessment findings need to be translated into actual engineering decisions
      - Improved assessment indicators do not necessarily represent higher business value
    - Model selection
      - What It Is
        
        Select the best model for practical application from the candidate model
      - Why
        
        No optimal model for all tasks
        
        Balance between model quality, cost and delay
        
        There is also a need to consider deployment modalities and other hard constraints
      - How to Do It
        
        Step 1: Screening of candidate models
        
        Distinguishing Hard Properties from Soft Properties
        
        Hard Properties
        
        Properties that cannot or cannot be changed
        
        Like licensing agreements, data privacy, deployment restrictions.
        
        Soft Properties
        
        Properties that can be improved by adaptation or optimization
        
        For example, in some cases, accuracy, delay
        
        Exclude models that do not satisfy hard limits
        
        Step 2: Open baseline first screening
        
        Selection of business-related benchmarks
        
        Checks whether the baseline is saturated
        
        Check training data contamination
        
        Check for correlation between different benchmarks
        
        Considering the weight of indicators in the light of operational importance
        
        Retain promising candidate models
        
        Step 3: Private assessment
        
        Re-enacting the current line created by step 3
        
        Test candidate model using the same business assignments
        
        Comparison of actual quality of models
        
        Comparison of costs and delays
        
        Step 4: Integrated options
        
        Exclude models that do not satisfy hard limits
        
        Selection of models to meet business quality requirements
        
        Balance quality, cost and delay in qualified models
        
        Establishment of deployment modalities
        
        Business model API
        
        Open weight model hosting API
        
        Open weight self-host model
        
        Comprehensive considerations
        
        Data privacy
        
        Data traceability
        
        Performance
        
        Functionality
        
        Cost
        
        Right of control
        
        Equipment end deployment