The trouble is that most of these contracts are signed without a standardized evaluation framework. This means many vendors will underdeliver. Some already have.
The vendors in this space aren’t necessarily bad actors, but the category is new and the technology is just obscure enough that procurement officers are often forced to evaluate platforms using criteria the vendors themselves wrote. It’s a classic procurement trap: Cities sign contracts based on “sunny day” demos, only to receive deliverables that don't match the sales pitch. By the time the gap is visible, the budget is spent and the vendor is already pitching to the next county over.
The problem is evident not just in street-level imagery like digital twins, but across the rapidly expanding smart cities sector as a whole. The U.S. smart cities market has roughly doubled in value over the last five years and continues to grow rapidly. Similar risks appear in other emerging technologies that cities are adopting, including Internet of Things sensors for traffic, air quality and parking; license plate recognition systems; and AI-powered predictive maintenance. Without rigorous scrutiny, these deals frequently lead to underperforming systems, vendor lock-in and wasted budgets.
This pattern is repeatable, but it’s also preventable. Asking potential vendors the following seven questions won‘t guarantee you pick the perfect partner, but they will ensure the wrong one has nowhere to hide.
1. Why are you pushing for a sole-source procurement?
2. What are your actual hardware uptime statistics?
Hardware fails. Cameras go down. GPS locks drop. Software pipelines break. Any vendor operating in the field has uptime data. Ask for it specifically. If your vendor responds to an uptime question with miles driven, images captured or parcels processed, they have answered a different question than the one you asked. That is not an accident. Demand the percentage of scheduled operational time the system was actually functioning and delivering data. You are evaluating a data company. If it cannot produce basic reliability data about its own operations, you should ask what else it is not measuring and what it is not telling you.
3. Is this hardware rated for our specific environment?
A system that works in San Diego will choke in a Marquette, Mich., winter or a Desert Hot Springs, Calif., August. Vague talk about “proprietary sensors” isn't a specification. Demand the thermal ratings.
Be careful: A camera housing rated for 150° F is not the same as a camera that functions at 150° F. Solar loading (the heat from direct sunlight) drives internal temperatures far higher than the ambient air. A vendor who can’t explain the difference between ambient rating and operational thermal performance has likely never successfully deployed in a real-world climate like yours.
4. Who owns the data after the ink is dry?
This is the most revealing question you can ask. Street-level imagery contains a goldmine of data on your residents, properties and public spaces. That data is valuable to insurance carriers, hedge funds and third-party developers.
Ask three things: Who owns the raw imagery? Can the vendor use your city's data for their own product development or third-party licensing? And what happens when the contract ends — do they delete the data, or do they keep a “residual” copy? Demand these rights be enshrined in the contract itself, not tucked away in a “Terms of Service” link that can be changed with a mouse click.
5. Are you charging us to access our own information?
Watch out for “per-seat” or “per-department” licensing. It’s a common revenue tactic: The city pays for the data capture, but the vendor retains the keys. Suddenly, public works, planning and code enforcement are all paying separate fees to view the same image of the same street, which the city paid to capture.
Before signing, demand unrestricted, citywide access. If a vendor claims per-seat licensing is “industry standard,” remind them that the standard should serve the city, not the vendor’s recurring revenue goals.
6. Can we see your entire client list?
Vendors always provide three happy references. Those contacts are often “power users” who get special treatment. Ask for the full list of every city they’ve worked with in the last three years. Then, call the cities they didn’t suggest. Ask them: Did the vendor hit their deadlines? Was the data messy? Would you sign that contract again? The vendor who refuses to share their full client list has already given you their most honest answer.
7. What does the “bad day” data look like?
Demos show the best-case scenario. You need to see the typical case. Request sample deliverables from actual production runs, not just curated pilots. What does the imagery look like in a dense urban canyon, during a rainstorm or on a road under construction? What is the error rate on parcel attribution? A vendor with a mature product will answer this easily; a vendor with “vaporware” will try to change the subject.
A FINAL NOTE ON AI
Every vendor will use the term “AI” like a magic wand. Treat it as a red flag that you need more specificity, not less. Ask what the models were trained on and who audits their accuracy. AI is a capability, not a guarantee of truth. The cities that find the most value in these platforms demand proof of performance before issuing payment.Erik Johnson is an urban economist and visiting assistant professor at the University of California, San Diego Rady School of Management. Erik’s research focuses on using AI and machine learning to extract economic insights from aerial and street-level imagery. Previously, he co-founded the University of Alabama’s AI working group and collaborated with civil engineers to modernize highway infrastructure assessments. His research has been featured in The New York Times and The Wall Street Journal.
