Trigger Spark Jobs based on various conditions

$10-30 USD

ที่ทำเสร็จแล้ว

โพสต์

เกือบ 6 ปีที่ผ่านมา

$10-30 USD

ชำระเงินเมื่อส่งงาน

Suppose there are two folders A and B. A folder will be loaded with Employee data files and B folder will be loaded with Department data files. Every minute, the java program should monitor these folders and trigger SparkA(based on folder A) or SparkB(based on folder B) programs(Spark-Submit or equivalent). After successful completion, the java program should move the processed files to a different location. The condition is either for every five minutes or for exceeding the size of the folder to more than 2GB.(Spark 2.1.1) You can use below code for reference: [login to view URL]

Hadoop

Java

Spark

รหัสโปรเจกต์: 16753314

เกี่ยวกับโปรเจกต์

6 ข้อเสนอ

โปรเจกต์ระยะไกล

ใช้งานอยู่ 6 ปีที่ผ่านมา

กำลังมองหาการสร้างรายได้ใช่ไหม?

ที่อยู่อีเมล

ประโยชน์ของการประมูลกับ Freelancer

กำหนดงบประมาณและกรอบเวลาของคุณ

รับเงินจากการงานของคุณ

อธิบายข้อเสนอของคุณในภาพรวม

ลงทะเบียนและเสนอราคาฟรี

หมอบหมายให้:

@sandeepfreax

Hi I am interested in this job. I believe that the skills and experiences I have gained at this position make me an ideal candidate for this position. I have extensive experience in implementing Spark-core, Spark-sql and Spark-Streaming. I have developed strong programming and debugging skills. I have completed dozens of PoC for my customers and received praise for the before time completion and that in Scala, Java and Python. Beside Apache Spark, I have also gained extensive experience in Jenkins and Automated / Unit Testing. My experience has afforded me the opportunity to become familiar with the some of the best used architectures in the programming. In particular, through my work I have become heavily involved in writing the good applications and debugging the high severity issues. It has also helped me in designing the best application model. I am always eager to learn about new emerging technologies in the world of programming and Devops Integration. I would love to bring the passion for programming and sharing the good practices with your application. I am confident that my experience in the Big Data and my programming skills in Scala / Java / Python will qualify me for consideration. If you would like, I can discuss my experience with you. I look forward to meeting with you and discussing my qualifications in more detail. PS : I am certified Spark, Scala and Java Developer. Beside that I held many certificates in different domains. Regards Sandeep

$55 USD ใน 4 วัน

0.0

(0 รีวิว)

0.0

6 ฟรีแลนซ์กำลังเสนอราคาโดยเฉลี่ย $73 USD สำหรับงานนี้

@Gaosong2017

Hello, dear How are you? I have experience in Hadoop in pyspark with python. I think that i can help you as well. Hope your contact me. Best Regards.

$100 USD ใน 1 วัน

4.9

(18 รีวิว)

5.2

@AmolZinjadeP

Hi, I have more that 4+years of experience in hadoop and data mining technologies like HDFS, MapReduce, python, Spark, Hive etc. Please review my profile for skills Contact me

$100 USD ใน 1 วัน