Data Engineer (Mid-Level)

About Us

We deliver innovative solutions that drive business transformation by streamlining operations, accelerating growth, and ensuring long-term success. From AI and cloud services to cybersecurity and custom application development, TekBay Digital Solutions LLC empowers your business to innovate and thrive in a constantly evolving digital landscape. We manage, optimize, and future-proof your IT infrastructure to help you stay ahead of the curve.

Key Duties and Responsibilities

As a Data Engineer, you will design, build, and maintain scalable data platforms that power analytics, AI, and business-critical applications. You will work closely with cross-functional teams to develop reliable data pipelines, ensure data quality, and support modern cloud-based data architectures.

1. Data Pipeline Development

• Design, develop, and maintain data ingestion workflows using Apache Airflow, including authoring DAGs, custom operators, custom hooks, and scheduling logic for new and existing client integrations.
• Write production-grade Python code to extract data from third-party SaaS APIs, including robust handling of pagination, authentication, rate limiting, and incremental/historical loads.
• Design and implement data transformation logic using Apache Spark (PySpark) to process structured and semi-structured data into a cloud lakehouse format (e.g., Apache Iceberg), with attention to partitioning, schema evolution, and performance.
• Build and maintain loading, embedding, and indexing flows for vector databases used in AI/ML and retrieval-augmented generation (RAG) workflows.
• Contribute to design reviews and propose improvements to pipeline patterns, frameworks, and shared utilities.

2. Data Quality and Operations

• Own day-to-day operations of assigned pipelines, including monitoring, on-call triage, and root-cause analysis of failures (timeouts, rate-limit errors, schema drift, upstream changes).
• Define and implement data validation, reconciliation, and quality checks using SQL and pipeline-level assertions.
• Design and improve retry, recovery, alerting, and SLA enforcement logic; drive incidents to resolution and produce post-incident reviews.
• Maintain and improve operational documentation, runbooks, and data flow diagrams; ensure they remain current as pipelines evolve.

3. Deployment and Infrastructure

• Containerize services and components using Docker, applying established patterns for configuration, secrets management, and image hygiene.
• Deploy and operate data services in Kubernetes-based environments across development and production clusters; independently debug pod, networking, and resource issues.
• Follow and help refine branching, code review, CI/CD, and release processes; review pull requests from peers and junior engineers.

4. Collaboration and Communication

• Contribute substantively to team stand-ups, sprint planning, design discussions, and retrospectives, including proposing scope, task breakdowns, and estimates.
• Communicate status, risks, and technical trade-offs clearly to the engineering lead, project managers, and client or partner counterparts.
• Translate ambiguous requirements into well-scoped engineering tasks; raise blockers early with proposed solutions.

5. Mentorship and Continuous Improvement

• Guide junior and associate engineers on day-to-day technical questions; provide constructive and actionable code review feedback.
• Contribute to onboarding materials, internal technical documentation, and team knowledge-sharing sessions.
• Stay current with the team’s data stack and the broader data engineering ecosystem; propose adoption of new tools or patterns where they provide clear benefits.

Required Qualifications

Education

• Bachelor’s degree in Computer Science, Computer Engineering, Software Engineering, Information Technology, Data Science, or a closely related discipline. Equivalent practical experience may be considered.

Experience

• 3–5 years of professional experience in data engineering, backend development, or data-intensive software engineering, including at least 2 years of experience building and operating production data pipelines.

Technical Skills

• Python: Strong proficiency, including idiomatic code, modular design, dependency management, testing (pytest or similar), and production debugging.
• Data Modeling: Working proficiency in building data warehouses with production-grade data modeling and data structuring practices.
• SQL: Strong proficiency, including complex joins, window functions, aggregations, and query performance tuning against relational databases.
• Apache Airflow: Hands-on experience authoring and operating DAGs, custom operators, custom hooks, and scheduling logic in a production environment.
• Apache Spark (PySpark): Working proficiency in distributed data processing, including transformations, partitioning, and basic performance tuning.
• Milvus: Working proficiency with vector databases for building production-grade RAG pipelines.
• MCP: Working experience building MCP servers.
• Data Formats: Practical experience with JSON, CSV, Parquet, and similar structured and semi-structured formats at scale.
• REST APIs: Demonstrated experience consuming third-party REST APIs in production, including authentication (OAuth, API keys), pagination, and rate-limit handling.
• Containers and Orchestration: Working knowledge of Docker (building, configuring, and debugging images) and familiarity with Kubernetes concepts (pods, deployments, services, ConfigMaps, and secrets).
• Version Control & CI/CD: Strong working knowledge of Git workflows; experience with CI/CD pipelines and code review processes.
• Linux / Command Line: Comfortable operating in Linux environments for development, debugging, and routine operational tasks.
• Cloud Platform: Hands-on experience with at least one major cloud platform (AWS, Azure, GCP, or IBM Cloud), particularly object storage and managed compute services.

Behavioral Competencies

• Independent problem-solving skills, with the ability to investigate, diagnose, and resolve issues with limited guidance.
• Clear written and verbal communication in English, including interactions with non-engineering and client stakeholders.
• Sound judgment when evaluating technical trade-offs within established patterns and frameworks.
• Ownership and accountability for assigned components, including their operational health and reliability.
• Collaborative mindset with the ability to give and receive feedback constructively while supporting junior engineers.
• Professional conduct in interactions with team members, clients, and partner organizations.

Preferred (Nice-to-Have) Qualifications

• Experience with other open data lakehouse formats such as Delta Lake or Hudi, including schema evolution and table maintenance.
• Hands-on experience with other vector databases (Pinecone, Weaviate, FAISS) and embedding technologies.
• Experience tuning Spark jobs for cost efficiency and performance at scale.
• Exposure to streaming or near-real-time data processing technologies such as Kafka, Spark Structured Streaming, or Flink.
• Experience with infrastructure-as-code tools (Terraform, Helm) and observability platforms (Prometheus, Grafana, OpenTelemetry).
• Experience operating in a client-facing or partner-facing engineering environment.

Why Join Us?

TekBay is an equal opportunity employer and does not discriminate based on race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. We carefully select candidates, test them for technical competency, and emphasize communication skills.

Core Benefits:

• Five-Day Workweek: Enjoy weekends off, promoting work-life balance compared to Nepal's usual six-day workweek.
• Office Perks: Stay energized with unlimited snacks, coffee, and wholesome meals throughout the day.

Leave Policies for full-time employees:

• 18 Days Annual Leave
• 12 Days Sick Leave
• Paid leave during probation period

Bonuses and Allowances:

• Anniversary Bonuses: Celebrate milestones with special rewards.
• Festival Bonuses: Extra financial support during festive seasons.

Professional Growth and Development:

• Higher Education Discount: Through our partnerships with education providers, employees receive discounts on further education.
• Paid Training and Certifications: Access fully funded courses and certifications

Workplace Culture:

• International Standards: Experience a diverse and inclusive workplace with global benchmarks.
• Team Building Activities: Regular events to boost collaboration and camaraderie.
• Referral Bonus: Earn rewards for bringing talented individuals into our team.

How To Apply?

We're constantly seeking exceptional individuals eager to bring their impressive talents to our team. If you're a dynamic force of skill and enthusiasm, join us in shaping our team's success. To apply, send your updated resume to careers@tekbay.digital.