Безопасная сделка - Гарант

Paying for service performance agree a reproducible load test

To pay against a performance promise, agree the environment, request mix, data volume, workload and pass conditions before testing. Measure delays together with errors and useful completed work. One fast response or average figure does not establish performance under the promised load. Preserve the scenario and original results so another reviewer can repeat the exercise. Test only within authorised scope on agreed infrastructure, protecting customers and third-party systems from uncontrolled effects.

Translate the promise into a measurable scenario

Works quickly does not identify a test. Name the user action, such as catalogue search, request creation or report generation. Define starting data and successful output. A server returning an empty response immediately has not demonstrated fast report generation merely because its response time is low.

Specify where timing starts and ends and which path it covers. Server response and complete interface loading are different measures. Do not substitute one for the other without agreement. If a promise concerns users in a particular region, measurement from a nearby server may not answer the relevant question.

Grafana k6 documentation demonstrates applying thresholds to test metrics. It does not prescribe suitable values for every product. Set numbers and acceptable variation around the actual task. A documentation example should not silently become a universal contractual quality standard.

Prepare a test passport

Record the application version, infrastructure, database volume, test records, load generator and time. Include cache, authentication and external-dependency settings. If some details cannot be disclosed, explain the boundary of reproducibility. The passport connects measurements rather than merely decorating a report.

Define the workload model: operations, proportions, timing and stage duration. Open connections do not necessarily equal useful user actions. Use the unit underlying the commercial promise. An incomplete scenario should not be described as an examination of the entire service.

Agree safety limits and stopping conditions, including excessive errors, production impact and unexpected expenses. Name a person authorised to stop. Testing does not justify real customer charges, broadcasts or order changes outside an explicitly prepared safe mode.

Create an acceptance matrix

For each operation, specify the measure, measurement boundary, threshold, acceptable error treatment and source. A hypothetical row might identify a test-product search, an agreed response-time measure, threshold X, required product presence and log Y. X is a value to agree, not a ready-made universal target.

A second row may cover a test request actually appearing in the database. Another can cover recovery after load returns to its normal level. Content, speed and recovery are distinct findings. The final decision should show which criterion passed and which did not.

Separate mandatory acceptance conditions from exploratory observations. A useful monitoring metric does not automatically become a rejection basis if never included in the scope. Equally, a critical condition should not disappear after a failed result. Version any changes to the agreed matrix.

Run the exercise in seven steps

  1. Customer and supplier approve the passport and criteria before launch. Identify people and authorised systems. Where an external service is involved, check its rules rather than treating your contract as permission to load someone else's infrastructure.
  2. Prepare safe records and isolate side effects. Confirm that requests perform the required work. Preserve scenario and configuration versions so reviewers can establish what actually ran.
  3. Run a short control stage. Check metric collection and whether the generator itself is a bottleneck. Tool failure should not be attributed automatically to the application, while missing measurements should not create a false pass.
  4. Execute the agreed profile and observe delays, failures and useful output together. Do not remove failed responses merely to make the remainder look fast. Exclusion rules need to be understandable in advance.
  5. Record external events and environment changes. Extra resources or disabled functionality during the run affect the result. Explain them before comparing the stage with the original promise.
  6. After correction, repeat affected scenarios under comparable conditions and examine return to normal operation. A short successful repeat does not erase history, but may establish a fix within its recorded scope.
  7. The authorised accepter records the decision for the identified version. Preserve inputs, configuration and limits. Payment should not imply a guarantee for every future workload or environment never tested.

Interpret results without false reassurance

Average delay can conceal slow responses for some users. Choose suitable distribution measures and explain their meaning. A selected percentile still does not describe everything: error rate, useful volume and observation adequacy remain relevant to the task.

Differences between runs do not necessarily show application deterioration. Data, cache, network or resources may differ. Compare the passports and investigate. Where causation remains unresolved, state the limit instead of assigning precise blame from a single graph.

In a teaching example, 1,000 attempts yield 900 correct results and 100 refusals. Reporting only the timings of the 900 successes is incomplete. These figures illustrate the population being measured; they do not set an acceptable failure percentage for a real project.

Preserve the reason for any excluded interval. A planned warm-up and an outage are not interchangeable simply because both produce inconvenient measurements. The agreed reporting rule should explain what remains in the acceptance population, and a reader should be able to recover the full observation history rather than see only selected favourable minutes.

Stop safely and investigate a failure

End the run when an agreed safety boundary is reached. Preserve logs and the stopping point. If the service fails the profile, establish whether the deviation reproduces and whether the profile matches the specification. Do not keep increasing load without authority simply to discover the absolute limit.

A delay caused by an external dependency still affects the user, but contractual responsibility and remedies require separate analysis. If test records were too small, correct the model and agree another run instead of treating the first number as a guarantee for production data.

If the outcome cannot be reproduced, request the scenario, records and environment details to an appropriate extent. A summary screenshot does not establish measurement provenance. A reviewer needs to understand the result even where commercially sensitive implementation details cannot be made public.

Attach payment to the defined measurement

Use the public transaction-stage outline to identify where review belongs, then agree your own test passport. The outline does not establish that an operator performs load testing on the customer's behalf.