Senior Software Engineer, Platform - Data + AI (Back-End)

Responsibilities

• Design and develop infrastructure and services to enable data pipelines for petabyte level scale and more.
• Design and develop abstractions over datastores such as Cassandra, PostgreSQL, Snowflake, etc.
• Design and develop file system abstractions over AWS S3, Azure Blobs, HDFS, etc.
• Design and develop connectors to various external data stores.
• Design and develop distributed system components for stream processing, queueing, batch processing, analytics engines, etc.
• Develop and maintain industry-leading, high-performance APIs for AL/ML applications.
• Develop and maintain features for distributed computations over large-scale data for ML workflows.
• Design and develop ML-specific data-systems such as feature stores and behavioral frameworks such as recommendation engines.
• Design and develop integrations with distributed computing technologies such as Apache Spark, Ray, etc. for data exploration and ML workload orchestration.
• Design and develop integrations with data analysis libraries such as Pandas, Koalas, etc.
• Develop and production AI/ML models for failure prediction, data schema inferencing, etc.
• Work on frameworks for performance, scalability, and reliability tracking over different components of a highly extensible AI/ML platform.
• Participate and provide insights in technical discussions.
• Write clean code following a test-driven methodology.
• Deliver commitments promptly following agile software development methodology.

Qualifications

• Bachelor of Science in Computer Science, Computer Engineering, or related fields.
• A minimum of 5 years of work experience in a fast-paced software company.
• Strong understanding of Computer Science fundamentals.
• High proficiency in coding with Java, C++, C#, or some other compiled language. Python would also be acceptable.
• Strong competency in object-oriented programming, data structures, algorithms, and software design patterns.
• Experience with version control systems such as Git.
• Experience with large-scale distributed systems.
• Experience with any public cloud platform (AWS, Azure, GCP).
• Some familiarity with distributed computing technologies (e.g., Hadoop, Spark, Kafka).
• Familiarity with managed versions of these technologies on public cloud platforms is also acceptable.
• Familiarity with technologies in the modern data science/analysis and engineering ecosystem (e.g., Pandas, Koalas).
• Good verbal and written technical communication ability to facilitate collaboration.
• Thrives in a fast-paced, dynamic environment and values end-to-end ownership of components.
• Intellectually curious and open to challenges.

Preferred Qualifications

• Advanced degree in engineering, sciences, or related field.
• Experience with Agile development methodology.
• Experience developing and working with REST and/or GraphQL APIs.
• Experience building scalable and reliable data pipelines.
• Experience with integration of data from multiple sources.
• Experience working with analytics and/or data processing engines.
• Experience developing distributed computation over large-scale data.
• Experience working with distributed computing frameworks (e.g., Hadoop, Spark, Kafka).
• Experience with data science/analysis libraries (e.g., Pandas, Koalas).
• Experience with task schedulers in distributed computing (e.g., Spark, Ray, Dask).
• Familiarity with machine learning workload orchestration in a distributed computing environment.
• Familiarity with workflow execution and/or optimization using DAGs, ideally for machine learning use-cases.
• Conceptual understanding of orchestration and resource provisioning systems (Kubernetes).

Pay & Benefits

California Base Pay Range $145,000—$187,000 USD
C3 AI provides excellent benefits, a competitive compensation package and generous equity plan.