Hi,
Hope you are doing well.
I like the way you provided the requirements.
I have extensive experiences in the integration platform interacting various systems who expose their data in the form of REST APIs, SOAP Services and pushing those data to various databases like SQL,No SQL(Cosmos DB,Storage table) by implementing various design patterns along with cross cutting feature of logging,monitoring covering retries,ratelimit,timeout,guranteed delivery,etc.
I can implement this in Azure with function app(C #.NET Core) with appropriate triggers(Timer/Blob/Queue)
taking care of time out issues,retry,ratelimit,guranteed delivery leveraging Entity framework.
I have few queries.
1)Is creating Sql transaction & its relational tables are in scope or you will take care of it.
2)Is technology stack is decided or yet to decide for example Azure or AWS or GCP or do you just wanted windows program so that you can schedule it in the Windows OS as windows scheduler.
Please kindly initiate a chat to take this forward.
Delivery:
1)Few page HLD
2)Sequence flow
3)Mapping document
4)Clean code following standard design patterns
5)Azure DevOps CI/CD pipeline for automated deployment.