Big Data Developer

Sydney, New South Wales Cognizant

Posted 3 days ago

Job Viewed

Tap Again To Close

Job Description

**Position Summary:**
We are seeking a highly skilled and experienced Big Data Developer to join our engineering team. The ideal candidate will have strong expertise in building and maintaining robust, scalable, and high-performance big data solutions using modern big data tools and technologies. You will be responsible for developing data pipelines, transform large volumes of data, and optimizing data infrastructure to enable real-time and batch processing for business insights and analytics
**Mandatory Skills:**
· Design, develop and maintain scalable and secure big data processing systems using tools like Apache Spark, Hadoop, Hive, Kafka etc.
· Build and optimize data pipelines and architectures for data ingestion, processing and storage from diverse source (structured, semi-structured, unstructured)
· Develop ETL/ELT workflows and data transformation logic using custom code in Python, Scala, Java and job scheduling tools such as Control-M.
· Translate business need/requirements into technical specifications, develop application strategy and high-level design documentation
· Involve on solution design discussions, competitive displacement, proof-of-concept engagements, solution demonstrations and technical workshops .
· Responsible to manage the development sprints effectively and efficiently from planning to execution to review in an agile development environment.
· Convert business knowledge into technical knowledge and vice - versa and able to translate those insights into effective frontline action.
· Responsible to build and manage end-to-end big data streaming/batch pipeline as per business requirements.
· Report and update the management/customer about the issues/blockers on time to mitigate risk.
· Collaborate with different teams, analysts and stake holders to understand data requirements and deliver efficient data solutions.
· Ensure data quality, consistency and security by implementing appropriate data validation, logging, monitoring, and government standards.
· Tune and monitor the performance of data systems, jobs and queries.
· Integrate and manage data platforms in Azure cloud environment.
· Mentor junior developers and support team best practices in code review, version control, and testing.
· Document system designs, data flows, and technical specifications.
· Proactively identify and eliminate impediments and facilitate flow.
· Experience in project management software (i.e., JIRA, Manuscript, etc.) to support task estimates and execution.
· Involving the engineering development projects and facilitates sprint releases.
· Create reusable assets / supporting artifacts based on the industry landscape
**Mandatory Knowledge & Qualifications:**
· Bachelor's or master's degree in computer science, data engineering, information technology, or related fields.
· 4+ years of experience in Big Data technologies.
· Should have experience in template-oriented data validation / cleansing framework.
· Strong programming skills in Python, Scala, Shell Scripting and Java.
· Extensive experience in data transformation and building ETL pipelines using PySpark.
· Strong knowledge in performance improvements of existing big data Streaming or Batch Pipeline.
· Proficiency in big data tools and frameworks such as Apache Spark, Hadoop, Kafka, and Hive.
· Strong proficiency in design and develop generic ingestion and streaming frameworks which can be used and leveraged by new initiatives/projects.
· Experience with NoSQL Databases - HBase and Azure Cosmos DB.
· Should have experience in optimizing high volume data load in Spark.
· Deep knowledge in Azure cloud platform.
· Experience in identifying complex data lineage using Apache Atlas.
· Strong knowledge of creating, scheduling and monitoring jobs using Control-M.
· Strong knowledge in Data analysis, Natural Language Processing (NLP), Machine Learning (ML) techniques, Data visualization.
· Deep understanding of data governance tool - Alex.
· Mandatory to have strong understanding of BFS domain.
· Build predictive models and machine-learning algorithms.
· Present information using data visualization techniques.
**Required Skills & Technology Stack:**
· Bachelor's or master's degree in computer science, data engineering, information technology, or related fields.
· 4+ years of experience in Big Data technologies.
· Should have experience in template-oriented data validation / cleansing framework.
· Strong programming skills in Python, Scala, Shell Scripting and Java.
· Extensive experience in data transformation and building ETL pipelines using PySpark.
· Strong knowledge in performance improvements of existing big data Streaming or Batch Pipeline.
· Proficiency in big data tools and frameworks such as Apache Spark, Hadoop, Kafka, and Hive.
· Strong proficiency in design and develop generic ingestion and streaming frameworks which can be used and leveraged by new initiatives/projects.
· Experience with NoSQL Databases - HBase and Azure Cosmos DB.
· Should have experience in optimizing high volume data load in Spark.
· Deep knowledge in Azure cloud platform.
· Experience in identifying complex data lineage using Apache Atlas.
· Strong knowledge of creating, scheduling and monitoring jobs using Control-M.
· Strong knowledge in Data analysis, Natural Language Processing (NLP), Machine Learning (ML) techniques, Data visualization.
· Deep understanding of data governance tool - Alex.
· Mandatory to have strong understanding of BFS domain.
· Build predictive models and machine-learning algorithms.
· Present information using data visualization techniques.
**Required Skills & Technology Stack:**
**Primary Skills:**
- Hadoop - **Expert**
- Apache Spark - **Expert**
- Apache Atlas - **Expert**
- Python - **Expert**
- Scala - **Expert**
- Shell Script - **Expert**
- PySpark - **Expert**
- Scikitlearn - **Expert**
- Matplotlib - **Expert**
- Azure Storage Explorer - **Expert**
**Database Skills:**
- HBase - **Expert**
- Hive - **Expert**
- Oracle - **Expert**
- SqlServer - **Expert**
- MySQL - **Expert**
- Cosmos DB - **Expert**
**Other Skills:**
- NLP - **Expert**
- ML - **Expert**
- Data Science - **Expert**
- Data Visualization - **Expert**
**Tools** :
- PyCharm/VSCode - **Advanced**
- Control M - **Advanced**
- Code Versioning Tool - Advanced
- Project Management tools (JIRA, Confluence, Service NOW) - **Advanced**
- MS Visio - **Expert**
- HP ALM/QC- **Expert**
**Salary Range:** $75,000-$85,000
**Date of Posting:** 25/August/2025
**Next Steps:** If you feel this opportunity suits you, or Cognizant is the type of organization you would like to join, we want to have a conversation with you! Please apply directly with us.
For a complete list of open opportunities with Cognizant, visit Cognizant is committed to providing Equal Employment Opportunities. Successful candidates will be required to undergo a background check.
#LI-CTSAPAC
Cognizant is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to sex, gender identity, sexual orientation, race, color, religion, national origin, disability, protected Veteran status, age, or any other characteristic protected by law.
This advertiser has chosen not to accept applicants from your region.

Machine Learning (ML) Consultant, AWS Professional Services

Sydney, New South Wales Amazon

Posted 2 days ago

Job Viewed

Tap Again To Close

Job Description

Description
AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and meet objectives with AWS Security Assurance Services. Our expertise and emerging technologies include AWS Partners, AWS Sovereign Cloud, AWS International Product, and the Generative AI Innovation Center. You'll join a diverse team of technical experts in dozens of countries who help customers achieve more with the AWS cloud.
As a Machine Learning (ML) Consultant, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will work directly with customers and innovate in a fast-paced organization that contributes to game-changing projects and technologies. You will design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience.
We're looking for ML Consultant capable of using ML, GenAI and other techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems.
Key job responsibilities
As a ML Consultant, you will:
- Collaborate with engineers to research, design, develop, and evaluate state-of-the-art ML and generative AI algorithms to address real-world challenges across industries
- Lead customer interactions to deeply understand the business problem, and help them in defining and implementing ML and generative AI solutions, as well as guide them on adoption patterns and paths to production
- Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and develop new solutions
- Create and deliver best practice recommendations, scientific artifacts, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholders
- Effectively communicate complex ML results and insights to diverse audiences, translating technical concepts for non-technical stakeholders while providing appropriate depth for technical audiences in both public-facing publications and internal knowledge sharing sessions
A day in the life
AWS Professional Services (ProServe) is a global team of consultants that helps enterprise customers achieve their business outcomes on AWS Cloud. ProServe consultants work with hundreds of the world's largest enterprises to help them deliver their cloud initiatives.
About the team
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating - that's why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
AWS values curiosity and connection. Our employee-led and company-sponsored affinity groups promote inclusion and empower our people to take pride in what makes us unique. Our inclusion events foster stronger, more collaborative teams. Our continual innovation is fueled by the bold ideas, fresh perspectives, and passionate voices our teams bring to everything we do.
Mentorship & Career Growth
We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.
Basic Qualifications
- BSc degree in Computer Science, Physics, Mathematics, Engineering or related, with 4+ years of experience in building machine learning models for business applications
- Scientific thinking and experience to plan, design, implement and evaluate experiments, with rigorous analysis capabilities.
- Experience with neural deep learning methods and machine learning
- Hands on experience building models with deep learning frameworks like PyTorch, Tensorflow, or Huggingface
- Experience in conveying rigorous technical concepts and considerations to technical and non-technical audiences
Preferred Qualifications
- Ability to think strategically about business, product, and technical challenges in an enterprise environment
- Hands on experience leading large-scale global data analytics projects aligned with business use cases.
- Experience in Data Quality management: Processes to measure and manage data quality, enforce data controls, and provide data assurance.
- Implementing AWS services in a variety of distributed computing, enterprise environments
- Customer facing skills to represent AWS well within the customer's environment and drive discussions with senior personnel regarding trade-offs, best practices, project management and risk mitigation
Acknowledgement of country:
In the spirit of reconciliation Amazon acknowledges the Traditional Custodians of country throughout Australia and their connections to land, sea and community. We pay our respect to their elders past and present and extend that respect to all Aboriginal and Torres Strait Islander peoples today.
IDE statement:
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit for more information. If the country/region you're applying in isn't listed, please contact your Recruiting Partner.
This advertiser has chosen not to accept applicants from your region.

Data scientist, australia - bcg x

Sydney, New South Wales Boston Consulting Group

Posted today

Job Viewed

Tap Again To Close

Job Description

part-time
Locations: Sydney PerthWho We AreBoston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact.To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate and digital ventures-and business purpose. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive. We Are BCG X We're a diverse team of more than 3,000 tech experts united by a drive to make a difference. Working across industries and disciplines, we combine our experience and expertise to tackle the biggest challenges faced by society today. We go beyond what was once thought possible, creating new and innovative solutions to the world's most complex problems. Leveraging BCG's global network and partnerships with leading organizations, BCG X provides a stable ecosystem for talent to build game-changing businesses, products, and services from the ground up, all while growing their career. Together, we strive to create solutions that will positively impact the lives of millions.What You'll Do Our BCG X teams own the full analytics value-chain end to end: framing new business challenges, designing innovative algorithms, implementing, and deploying scalable solutions, and enabling colleagues and clients to fully embrace AI. Our product offerings span from fully custom-builds to industry specific leading edge AI software solutions. As a Data Scientist, you'll be part of our rapidly growing team. You'll have the chance to apply data science methods and analytics to real-world business situations across a variety of industries to drive significant business impact. You'll have the chance to partner with clients in a variety of BCG regions and industries, and on key topics like climate change, enabling them to design, build, and deploy new and innovative solutions. Additional responsibilities will include developing and delivering thought leadership in scientific communities and papers as well as leading conferences on behalf of BCG X. Successful candidates are intellectually curious builders who are biased toward action, scrappy, and communicative. We are looking for talented individuals with a passion for data science, statistics, operations research and transforming organizations into AI led innovative companies. Successful candidates possess the following: Comfortable in a client-facing role with the ambition to lead teams Likes to distill complex results or processes into simple, clear visualizations Explain sophisticated data science concepts in an understandable manner Love building things and are comfortable working with modern development tools and writing code collaboratively (bonus points if you have a software development or Dev Ops experience) Significant experience applying advanced analytics to a variety of business situations and a proven ability to synthesize complex data Deep understanding of modern machine learning techniques and their mathematical underpinnings, and can translate this into business implications for our clients Have strong project management skills What You'll Bring Preferably master's degree or bachelor's in computer science, data science, mathematics, statistics, engineering, or a related field1-3 years of relevant work experience in data science, machine learning, artificial intelligence, or big dataProficiency in Python and other programming languages and tools for data analysis and machine learningExperience in using machine learning techniques such as regression, classification, clustering, natural language processing, computer vision, etc.Familiarity with industry-specific skills such as retail analytics, healthcare analytics, financial analytics, etc.Boston Consulting Group is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, age, religion, sex, sexual orientation, gender identity / expression, national origin, disability, protected veteran status, or any other characteristic protected under national, provincial, or local law, where applicable, and those with criminal histories will be considered in a manner consistent with applicable state and local laws. BCG is an E - Verify Employer. Click here for more information on E-Verify.
This advertiser has chosen not to accept applicants from your region.
Be The First To Know

About the latest Data scientists Jobs in Sydney !

 

Nearby Locations

Other Jobs Near Me

Industry

  1. request_quote Accounting
  2. work Administrative
  3. eco Agriculture Forestry
  4. smart_toy AI & Emerging Technologies
  5. school Apprenticeships & Trainee
  6. apartment Architecture
  7. palette Arts & Entertainment
  8. directions_car Automotive
  9. flight_takeoff Aviation
  10. account_balance Banking & Finance
  11. local_florist Beauty & Wellness
  12. restaurant Catering
  13. volunteer_activism Charity & Voluntary
  14. science Chemical Engineering
  15. child_friendly Childcare
  16. foundation Civil Engineering
  17. clean_hands Cleaning & Sanitation
  18. diversity_3 Community & Social Care
  19. construction Construction
  20. brush Creative & Digital
  21. currency_bitcoin Crypto & Blockchain
  22. support_agent Customer Service & Helpdesk
  23. medical_services Dental
  24. medical_services Driving & Transport
  25. medical_services E Commerce & Social Media
  26. school Education & Teaching
  27. electrical_services Electrical Engineering
  28. bolt Energy
  29. local_mall Fmcg
  30. gavel Government & Non Profit
  31. emoji_events Graduate
  32. health_and_safety Healthcare
  33. beach_access Hospitality & Tourism
  34. groups Human Resources
  35. precision_manufacturing Industrial Engineering
  36. security Information Security
  37. handyman Installation & Maintenance
  38. policy Insurance
  39. code IT & Software
  40. gavel Legal
  41. sports_soccer Leisure & Sports
  42. inventory_2 Logistics & Warehousing
  43. supervisor_account Management
  44. supervisor_account Management Consultancy
  45. supervisor_account Manufacturing & Production
  46. campaign Marketing
  47. build Mechanical Engineering
  48. perm_media Media & PR
  49. local_hospital Medical
  50. local_hospital Military & Public Safety
  51. local_hospital Mining
  52. medical_services Nursing
  53. local_gas_station Oil & Gas
  54. biotech Pharmaceutical
  55. checklist_rtl Project Management
  56. shopping_bag Purchasing
  57. home_work Real Estate
  58. person_search Recruitment Consultancy
  59. store Retail
  60. point_of_sale Sales
  61. science Scientific Research & Development
  62. wifi Telecoms
  63. psychology Therapy
  64. pets Veterinary
View All Data Scientists Jobs View All Jobs in Sydney