Property Finder fixes evaluation bias and optimises its Bedrock model
What happened
Property Finder found that 96.8% of its evaluation labels came directly from predictions by existing vision-language models rather than human verification, structurally disadvantaging any alternative model in comparisons. The team then re-annotated 356 UAE images by hand, switched to multi-label classification and combined prompt iteration from v0 to v27, ultimately selecting a new model on Amazon Bedrock with higher accuracy and 86% lower cost.
Written by AI from the reports · updated 3 h ago
Timeline
Follow the reports to see the event from each side.
- AWS Machine Learning BlogFix evaluation bias to unlock better models on Amazon Bedrock
Property Finder found that 96.8% of their vision-AI evaluation labels were inherited from the incumbent model, hiding its true accuracy. See how they built unbiased ground truth, applied evolutionary prompt optimization, and selected a more accurate, cost-effective model on Amazon Bedrock, cutting cost by 86% across 11 image-quality attributes.
Heat of this event
Current heat 9·Peak in the comparable range 9（Oct 28 03:00）·Change over 24 hours in the comparable range –
The trend compares only the same accounts observed without a break, so its range can be narrower than the current heat count. Move the pointer or tap the chart to see each hour; with a keyboard, use the left and right arrow keys.