Financial Data Import, Verification and Process Improvement

A financial company that sends a market update at 2 am PT (Pacific Time) that lists modifications of thousands of instruments for 20,000 accounts. This import needs to be done perfectly every night. Right now the processes break about 5% of the time (about once a month). Monitor the import daily with extremely high reliability. Working in a time zone that has normal business hours at 2am and I'll expect them to work through any import issues until they are fixed on any day they happen to break. So, if you're in Dublin, Irland that would be perfect. One year contract. Their job has two parts: 1) Daily verify the import happens properly. At 2 am PT (11 am in the Ukraine) they verify that the import has begun and they send out an email on its successful completion. Or a status report when it fails. They debug and fix any failures (Initially with Andy's guidance). 2) Modernization of Daily Folio Import. A. Stage 1: convert import to be JSON driven. JSON is a name/value pair, a self-describing language which is an industry standard for large datasets like this (large-scale databases like Elastic Search and Hadoop present data in JSON rather than, relational, tabular or separated values. B. Stage 2: investigate more current Folio formats and ultimately switch to them after a period of verification of identical results with the new format.