|
About the Role:
As a Data Asset Developer, you will play a critical role in designing robust, scalable data architectures that power business intelligence and analytics across the cybersecurity landscape. You will develop data models that meet the reporting and analytical needs of the cyber team while working closely with data scientists and analytics/BI engineers to optimize the use of the data models, ensuring they are efficient and scalable.
You will collaborate with data consumers to understand the requirements, design and deliver the data models, implement best practices for data governance, and help set standards for data pipelines and solutions, ensuring seamless integration.
Principal Accountabilities:
Data Architecture and Modelling
- Design and implement high-quality, scalable data models that ensure the integrity, security, and accessibility of Cybersecurity Data Assets.
- Collaborate with analytics/BI engineers to ensure these models are used effectively.
- Produce high quality documentation and metadata.
- Operate the data governance processes to publish and manage the data assets in accordance with data controls.
Databricks Expertise
- Design and implement data architectures using Databricks notebooks and Unity Catalog.
- Build robust, scalable data pipelines within Databricks.
Required Skills and Qualifications
The successful candidate will be a credible leader, possess skills and experience within one or more of the following areas, and demonstrate a willingness to learn additional skills via certification and/or on-the-job learning where required.
- Overall, 5 years of proven experience in data architecture, modelling, and database design, including star & snowflake schemas.
- 3+ years of experience with Databricks notebooks, Unity Catalog, and building scalable data pipelines using PySpark/Spark, MySQL.
- Hands-on experience working with multi-terabyte data sets, optimizing for performance and cost-efficiency.
- In-depth understanding of ETL processes, and data warehouse architectures.
- Familiarity with Azure cloud platform and data solutions.
- Experience with data governance, security, and compliance practices.
- Excellent communication, problem-solving, and collaboration skills.
Preferred Qualifications
- Experience with data Lakehouse architectures and modern data engineering practices.
- Knowledge of machine learning pipelines and advanced analytics using Databricks.
- B.Tech, MCA, or MTech in Computer Science, Information Technology, Engineering, or related fields is a plus.
Nice to Have:
- Relevant certifications in Databricks, or cloud platforms.
- Broad knowledge of cybersecurity principles and disciplines.
- Experience working in or with cybersecurity teams.
|
