Redshift, Athena, Spectrum, Glue, AWS, Data, Lake, S3, QA, MySQL, RDS, Redshift Data Warehouse, AWS SageMaker, Python, DynamoDB, RestFul API, Near Real Time ETL, ELT Into Redshift.
Role Purpose
To play an integral part supporting the migration from the current AWS MySQL RDS to near Real Time Redshift Data Warehouse. To help expand our Data Lake using Athena, Spectrum and Glue to ensure a secure, robust Data Lake environment which helps to support ML using AWS SageMaker
Please see below ESSENTIAL SKILLS and only apply if you have them. We cannot progress any application without that needs list!
Based on Customer site North Wales - Commutable from Shrewsbury, Stoke On Trent, Warrington, South Liverpool, Wrexham, Oswestry, Crewe, Nantwich, Telford
Key Responsibilities
- Build and support enterprise-level data warehousing within Redshift.
- Ensure that IAM is used to control access at a fine grain - ensuring horizontal and vertical security is implemented within data warehouse and data lake.
- Create near Real Time feeds into Redshift from S3 using AWS Glue.
- Build a Translation/QA area within Redshift to hold data failures which will be exposed via dashboards to the QA team.
- Optimise and enhance our Data Lake using best practices.
- Support existing DataOps Team and mentor/share knowledge.
- Deliver supporting automated processes (ie SAR, data retention, right to erasure) to ensure the data estate complies with best practice and regulatory requirements.
- Critically appraise existing Data Warehouse/Lake and suggest improvements.
Essential Skills & Experience
- Very experienced with Redshift, Athena, Spectrum and Glue.
- Experienced with IAM integration with Redshift, Spectrum and Athena.
- Redshift administration.
- Experience of Kimball.
- Experience of near Real Time ETL/ELT into Redshift.
Desirable Skills & Experience
- Experience of Python
- Understanding of RESTful API
- Experience of GDPR/PECR/DPA
- DynamoDB including streams/TTL etc.
