Python tools need some improvements

Hi ! I got few simple tool written in python ! Problem in them is that those tools are very slow ! I need decent performance improvemnt for them ! Tool 1: [login to view URL]!M00BVCxS!n4UbvDvv-z4PMTE9iLcdrAWBH3F_tVgB5KGL_AmufcQ Consits of few modules: - [login to view URL] - add multithreading - improve algorithm for better performance if possible - maybe using indexed database instesd of file as data source would help? - [login to view URL] - add multithreading if possible - make as fast algorithm as possible - should work with files like 50GB but not eat more than 75% of computer RAM - [login to view URL] - already multothreaded ! but still very slow !! need decent improvemnt because LSA* scripts are most important to me Tool2: [login to view URL]!8wEwAZQS!s8EAQK27Lh1IiA5XMbjGv1oISjbmSyOXKwIK5Phbatw removes duplicates from parallel sets of texts. Now it works but problem is that I want to process 300GB data and the tool tries to load all into memory that needs to be solved. Also introducing multithreading is needed. Other algorithmical improvements are not needed but well seen if possible. Tool3: [login to view URL]!E0tkUKDY!4CxlRzz3omo2ux3CpUjtsqls0wWw_QLkPbEGgHdG4uI Works good, but too slow. I would need to add multhreading (but the tools should not consume more that 75% of PC RAM). Other algorithmical improvements are in my opinion requred if possible.