Suppose two suppliers' pump samples differ by only a few percent on a buyer's duty. The familiar supplier is tested first, followed by the challenger. Every repetition uses that order. Before turning the result into a purchase decision, consider whether it can distinguish a small real difference from the order and the operator's expectations. An appropriate masked and alternated comparison may help. It does not convert a few samples into evidence about an entire production lot.
Mask only the identity that is not needed for the test
The test team still needs the exact supply, safe setup, wet end, connection and control conditions. Do not remove required markings or withhold safety information. A custodian can allocate neutral identifiers while preserving each pump serial number, revision and supplier lot. Present reports to the reviewer by those neutral identifiers. When the product shape or handling makes it obvious which sample is which, do not call the operator fully blinded. A reader who receives comparable raw records without supplier names can still assess the results using the pre-agreed rule.
Write the duty, input-power boundary, tolerances and treatment of unexpected readings before starting. Do not grant one unit extra repeats while keeping the other unit's first result unless the protocol already explains why. Record the instruments, water level and temperature, fixture and start conditions, and the criterion for stable readings. Qualified personnel must identify the correct electrical version and perform live wet bench work under the approved procedure. An unsafe condition calls for stopping, regardless of how it affects the comparison.
Balance the order and check the record for drift
As an illustrative test block, run A, B, B and A under the same target conditions, returning each sample to the specified starting state when required. A separate block could reverse the order. Do not insist on this particular order for every test; choose a design appropriate to the task. The principle is to avoid assigning all favorable early conditions to one sample and all later conditions to the other. A check pump or repeat reading can help reveal a bench change. If output moves with the order of the runs and a reference also shifts, look at the bench and fixture before interpreting the difference as a design result. Alternation does not remove a sudden change, nonlinear drift or conditions introduced while swapping units.
For illustration, A appeared two percent better in the first sequence. Under the revised comparison, the repeats show results on both sides and the difference is near the experimental uncertainty. Report that the current tests do not distinguish them reliably, rather than selecting the single reading that favors either brand. Keep the original record, including the reasons a reading was excluded or repeated. Unmask the supplier after comparing under the rule, with serial and batch mapping retained for traceability.
Tie the decision to the strength of the evidence
A repeatable difference may justify checking more independent pumps of the actual production versions. If it cannot be distinguished yet, first apply the agreed minimum requirements and compare verified supply, accessories and support. An unresolved small difference is not a reason to change the classification of valid results after the test. Report how many separate units were tested and how many repeats each had. Five readings on the same pump say something about repeatability; they do not make five pumps. The buyer should be able to reconstruct the order, measurements and conditions later without being told which supplier was preferred.
Frequently asked questions
Is deleting the brand name from the final report enough for blinding?
Set who knows the mapping before measurement and evaluation. Keep the safe test information and traceable identification available.
Can reversing the order once correct all drift?
No. Check the bench and data for the nature of any change and apply the agreed uncertainty or retest procedure.
Do three readings on each of two samples make six independent samples?
No. There are two physical samples and six readings. Report those separately.
