Recent media reports have alleged that New York City has been overpaying for its implementation of procurement software suite provider Ivalua in NYC’s procurement transformation efforts. The reports have used a watchdog group’s analysis that has tried to compare the seemingly high price tag of the NYC implementation of Ivalua to a smaller implementation of the city of Dallas by a more niche software provider named Bonfire. Ivalua and Bonfire are two procurement software providers that Spend Matters covers within the broad procurement provider ecosystem.
The headlines appeared suspicious, and we decided to take a deeper look at the projects and the providers in question. Our analysis indicates that one report’s direct comparison of these costs is misleading and flawed.
An article by the Checks and Balances Project notes that NYC is paying about $45 million to overhaul its procurement operation citywide and that Dallas contracted with another firm for just $50,000. The article also mentions that NYC’s subcontractors on the project were changed by the city from one consulting firm to another, with KPMG being replaced with Accenture.
We have a few takeaways from the group’s conclusions, and our analysis indicates that the NYC deal could easily pay for itself and be a boon for the city.
First, it’s certainly fair game to understand the process by which the initial competitive sealed bidding process worked to select Ivalua, the eventual winner, especially since it was a public tender. It is also fair to understand how the city chose KPMG, why it subsequently replaced KPMG with Accenture (also discussed in a NY Daily News story), and how the tendering process for those services was also executed fairly and transparently.
Second, and even more importantly, it’s also fair to ask whether the large price tag for the initial deal and subsequent deal is justified and delivering value to the taxpayers of NYC.
However, while watchdog firms can provide a valuable service to the public, more input from experts in the procurement technology field can help make these assessments fairly and accurately.
In this case, the group’s contention is that the city of New York is getting a raw deal. Unfortunately, its conclusions are reached based on faulty analysis. NYC’s massive procurement transformation and technology implementation project for end-to-end procurement using Ivalua is completely different in scope and scale to Dallas’ small deal for a hosted e-sourcing tool from Bonfire. This isn’t just comparing apples to oranges, it’s comparing a watermelon to a grape.
Both providers do excellent work based on the unbiased practitioner reviews that we collect as part of our evaluation of them. But, Ivalua and Bonfire are extremely different, and so are these public-sector implementation projects. The watchdog firm’s question of: “Is it reasonable that NYC’s spending on an e-procurement system should be nearly 340 times that of Dallas?” is based on an apples-to-oranges false equivalence. That’s a shame because this is a project that could actually deliver a massive and defensible return on investment. We can help shed some more light on this.
Apples and Oranges
In terms of these two vendors, we compare roughly 60 of these providers across hundreds of requirements using our SolutionMap assessment methodology, which is by far the deepest competitive intelligence database for procurement technology solutions in the industry. Bonfire is a top-notch, low-cost “nimble” solution for electronic sourcing. For a graphical comparison of it versus similar peers, check out our “Nimble” Solution Map for Sourcing in the chart below and for free on our site here.
Bonfire is a great example a strategic sourcing application that is geared toward power users in procurement working with stakeholders and suppliers to negotiate contracts. Ivalua on the other hand is a market-leading full e-procurement application suite that performs spend analytics, strategic sourcing, contract management, transactional purchasing (which is deployed to all employees who need to request and shop for items and services), and vendor management. It’s best viewed competitively within our “Configurator” persona for the Source-to-Pay area here and in the graphic below.
Please note that these graphics are highly dependent on a customer’s requirements, and the graphics above only represent the results based on the weightings creating by our analysts in the respective solution areas for a typical customer “persona.” These vendor placements will change based on a unique customer’s requirements (which can be obtained via two of our premium selection services).
Apples to Apples
Now, let’s get back to that purported 340x cost difference …
- First of all, per this NYC government website page, NYC has roughly $20 billion in spend across its 40 agencies. The city of Dallas, which uses Bonfire, reports that it has an annual spend of $876.9 million (cited here). So, NYC has basically 22X of the spend scope.
- Next, the NYC project includes total costs for implementation, not just the software licensing. For every dollar spent on annual software subscription, there is roughly $10 spent on services. If you look at the software subscription costs within the change order document cited in this article (which has all sorts of excellent detail), you’ll see a $4.8 million line item price tag for the five-year software subscription, and the remaining $40 million of the $45 million contract is for implementation/support services. So that’s another 10X difference, which combined with the 22X, explains 220X of the 340X difference. The final adjustment deals with scope because the NYC project covers roughly three times the process scope than the city of Dallas. Suddenly, NYC is actually looking more efficient!
- Let’s do a few “back of the napkin” stress tests here. First let’s compare the annual subscription costs normalized to (i.e., divided by) the annual supplier spend. For Bonfire/Dallas, it’s 0.576 basis points, and for Ivalua/NYC, it’s 2.4. But, Ivalua/NYC is about at least 4x the project scope, so they look about the same.
- Let’s do another stress test. If you’re going to compare the cost of the software of these two cloud-based software systems on a normalized basis, you might typically look at the cost per user per year:
- For Bonfire, the $3,300 price tag ($50,000 divided by 15 licenses) is an annual subscription per “power user” in procurement per year (casual users in the business and with the suppliers don’t pay anything) — not a cost per department like the article states.
- For Ivalua, the $4.8 million equivalent software subscription cost on the five-year deal works out to $960,000 per year. NYC presented at a conference last year that they’d gone live with 6,000 users during phase 1 (vendor management) of the three-phase deal, so conservatively speaking, Ivalua would cost $160 per user per year.
So, the Ivalua system in NYC actually seems 20X cheaper per user than the Bonfire system in Dallas! However, this is not the best comparison because Bonfire as stated before is built for a small number of power users rather than for a multi-product suite that is also being rolled out to a massively larger number of employees. Based on the change order document cited earlier, phase 2 was slated to be implemented a few months ago, and our sources indicate that it has indeed gone live. Phase 2 is the requisition-to-order phase, which obviously touches a larger portion of the total 325,000 NYC employee base that might access the requisitioning system (we don’t know yet the exact user count here). Regardless, the cost per user will obviously decrease even more. We’re in the process of securing an interview with NYC procurement leadership, so stay tuned for more on some of these numbers and insights on this massive implementation.
Again, it’s really important to compare apples to apples if you want to make judgments on the value being received by these municipalities.
From Cost to Value
Bottom line, based on our analysis, NYC seems to be doing pretty well cost-wise, and the change order document is very commercially thorough and well laid out (e.g., granular milestone-based deliverables, not-to-exceed thresholds, and so forth).
The watchdog article also incorrectly calls out how the Ivalua software as not yet live two years after the deal was signed, and it didn’t explain why the project increased from $30 million to $45 million — basically implying that is was simply a cost overrun from poor management. First of all, two of the three phases have indeed gone live. In terms of scope increase, if you go through the document, you’ll see that the scope increased because the Department of Education wanted to get in on the implementation and because of additional functionality and integrations to other existing systems. You’ll also see that the first phase of the implementation is design, which is work geared to ensuring that the eminently configurable Ivalua solution could actually be stretched to its limit to support the complex requirements of 40 government agencies in the largest city in the U.S. — and doing so with a continually updated “vanilla” cloud-based solution rather than a customized solution that would invariably have a higher total cost.
Ivalua is actually just releasing its public sector application (for state/local government) out to the market, and the “battle hardening” done in NYC will certainly prove valuable because as Frank Sinatra sang: “If I can make it there, I’ll make it anywhere!”
The Bigger Picture
For NYC, the risk here is clearly not price risk for the software. Implementation is where the real risk lies. Accenture and Ivalua have their work cut out for them, and it’s definitely a strategic and high-stakes implementation for both firms. However, the potential reward is equally great. Even if NYC can use this best practices-based software to drive a conservative figure of 1% annual incremental savings on that $20 billion of spend, that’s $200 million of annual hard-dollar benefits — making the $9 million annual investment on the five-year deal look like chump change. But again, execution will be key.
Still, it’s also not just about the cost savings that can be reinvested to serve the citizens better and/or lower their taxes. Indeed, the NYC citizens deserve to be better served by the city based on various metrics (this is a common theme that we hear from government agencies themselves), and readers interested in public procurement metrics can check out this video from our sister organization Public Spend Forum.
New York’s comptroller summed up the problem in an annual report on contracting with the city, saying that the approval process has good intentions but can lead to long delays. Some vendors, including nonprofits, must wait months to get paid. But since payments are the end of the upstream source-to-pay process, the upstream sourcing process also likely needs a major overhaul (with RFP cycle times measured in many months too). Public sector sourcing/bidding (or “tendering” if you like) is notoriously complex based on a patchwork of policies, statutes, inertia and bureaucratic tribalism — and I’m sure NYC is no different.
So, the sourcing implementation at NYC will likely be challenging, but the city has got top-notch software and consultants (and a few successful phases under its belt). Now NYC procurement “just” needs to execute on the sourcing phase — and needs support rather than uninformed analysis (even if well intentioned). We feel that NYC’s implementation isn’t just a technology automation project, but rather a transformation effort to bring older processes and practices into the modern age. New York taxpayers should be proud that they’ve got some government leadership working hard to get more bang for the $20 billion they spend with contractors — and using technology to provide efficiency, transparency/oversight and commercial excellence to deliver that. We pay attention to the best practices and best technologies in the private sector and can tell you that you’re on the right track!
Hopefully the next story about NYC procurement in the business press will be some of the proven results generated by applying these commercial best practices and technologies to make the best use of limited tax dollars. Proper scrutiny of such large projects is certainly appropriate, but we feel that such transformations should be encouraged rather than casually evaluated and cast in a poor light.