Senior Data Engineer

Job Duties And Responsibilities

• Design, develop, and support cloud-native systems that enable automated deployment and management of applications, including collecting, extracting, transforming, and making available large volumes of data to support observability, system analysis, and service availability.
• Mentor junior engineers and new team members, and apply technical expertise to challenging programming and design problems.
• Perform processing of streaming data from telecom sources (5G/4G network logs, IoT sensors, edge devices) using platforms like Kafka, Spark Streaming, and Flink to enable near-real time AI analytics.
• Contribute to large scale, data driven initiatives to drive enhanced visibility and quality of the wireless network.
• Design, build, and maintain scalable data pipelines to collect, process, and integrate large-scale telecom data (e.g., call detail records, network telemetry, OSS/BSS data, customer usage patterns).
• Develop high-volume, low-latency, data-driven solutions utilizing current and next generation technologies to meet evolving business needs.
• Establish standards and guidelines for the design & development, tuning, deployment and maintenance of information, advanced data analytics, and text mining models and physical data persistence technologies.
• Acquire big data input from numerous partners; key technologies may include Python, Mimir, Loki, Prometheus, and Kafka.
• Normalize complicated data sources to convert potentially unusable data into a format that can be efficiently used by software and/or employees; key technologies may include Spark, Scala, Lambda.
• Maintain a CI/CD pipeline for data software to ensure high quality and low time to market; key technologies may include Gitlab, Airflow, Ansible.
• Automate infrastructure provisioning and orchestration for data and AI workloads using CI/CD pipelines and Infrastructure as Code (Terraform, CloudFormation).

Skills, Experience And Requirements

• Bachelor’s degree or U.S. equivalent in Computer Science, Computer Engineering, Information Technology, or related field, plus 4 years of professional experience as a Software Engineer, Software Developer, or any occupation/job title/position applying ETL concepts to integrate, normalize, and process data from multiple sources to support system monitoring, observability, and analysis.
• Experience in collaborating with cross-functional technical teams to gather requirements, clarify system needs, document solutions, and resolve technical issues or ambiguities.
• Experience in developing and supporting real-time or near-real time data streaming solutions, including use of streaming technologies (including Kafka) to ingest and process network telemetry or operational data.
• Experience in designing, deploying, and supporting cloud-based data platforms and infrastructure.
• Experience in working with telecommunications cloud infrastructure, including deployment and support of telco cloud architectures, network functions, and distributed containerized cloud platforms for 5G or edge systems (including WindRiver Studio Cloud Platform).
• Experience in using data processing engines, including Apache Spark, to transform and analyze large-scale datasets.
• Experience utilizing monitoring, observability, and visualization tools, including Grafana, Prometheus, and PostgreSQL, and application or services monitoring tools (including Datadog), to monitor system performance, availability, and fault conditions.
• Experience using version control and collaboration tools, including GitLab, to manage code artifacts and support CI/CD workflows.
• Experience using cloud platforms, including AWS (including S3, Athena, CloudWatch, Redshift, AppFlow, and CodeCommit), to collect, process, store, and monitor large volumes of operational and network-related data.