From FTP to API: how to modernize a legacy data connection
Almost every manufacturing company has them: connections that were once “temporary” and never replaced. A nightly batch job exports CSV, sends it via FTP to a logistics partner, who manually imports it into their own system. It has been running for years. Nobody dares touch it.
Until it breaks.
The concrete problem
A client in the food manufacturing industry had exactly this kind of connection. Every night, the ERP exported order data as CSV, placed it via FTP on a shared server, and the logistics partner manually imported the file the next morning. The connection was originally built in an afternoon as a stopgap. That was eight years ago.
After migrating to Business Central 365 in the cloud, direct database export was no longer available. The old connection (built on the assumption that you could access the database directly) had to be rebuilt. The question was: do we rebuild the same thing, or take a fundamentally different approach?
Why batch connections are dangerous
The problem with nightly batch integrations is not that they are slow. The problem is that they hide uncertainty.
An FTP connection has three failure modes you only discover when it is too late:
- Silent failures. The file is generated but contains incomplete data. There is no validation, no schema check, no confirmation. The partner imports whatever they receive.
- No feedback loop. If the upload fails, nobody notices until the next business day. There is no retry, no alert, no audit trail.
- Version drift. The source file changes structurally (an added column, a different date format), but the receiving end parses the old format. No error, just wrong data.
In practice this means: if something goes wrong on Tuesday, you discover it Wednesday. If you are unlucky, Friday. And if it silently delivers incorrect data, perhaps never.
The API-first approach
Instead of rebuilding the FTP connection, we set up an event-driven architecture in three steps.
Step 1: OData as the data provider
BC365 offers a standard OData layer for exposing business data. Instead of generating nightly CSV, we expose the relevant entities as API endpoints. Data is no longer pushed at a fixed time; it is retrieved when the consumer needs it.
Step 2: Azure API Management as the gatekeeper
Between BC365 and the partner, we placed Azure API Management (APIM). This delivers in one go what the FTP connection entirely lacked:
- Authentication via OAuth2/JWT: no more shared passwords
- Rate limiting: protection against overload
- Request logging: every call is recorded with timestamp, payload and response
- Version management: add new fields without breaking the existing integration
Step 3: Webhooks for event-driven notification
The logistics partner no longer needs to poll. When a new order or status change occurs, the system sends a webhook notification. The partner receives an event, retrieves the current data via the API, and processes it, within seconds rather than within a day.
What this delivers
| Aspect | FTP batch | API integration |
|---|---|---|
| Data freshness | 24 hours old | Realtime |
| Error detection | Next business day | Within seconds |
| Authentication | Shared FTP password | OAuth2/JWT per partner |
| Audit trail | None | Every transaction logged |
| Version management | Non-existent | Built into APIM |
But the most important difference is not technical. It is operational. Errors become immediately visible. There is no longer a nightly window in which data can silently be lost. And when something fails, you know exactly what, when and why.
The lesson
Replacing legacy connections does not have to be a big bang. The FTP connection kept running as a fallback while we built the API route. Only when the partner confirmed the new integration was stable did we switch off the old connection.
That is the difference between modernizing and rebuilding. You do not replace everything at once; you lay a better route alongside the existing one, validate, and then switch over.
The connections that run the longest are often the connections nobody feels responsible for. That does not make them unimportant. It makes them dangerous.