What’s a SQL ETL developer?

Aside from becoming a mobile app developer, I am a full-time SQL ETL developer at an insurance company; I often get asked ‘What do I do?’ or ‘What is that?’. So I figured I would rite a quick high level view post about it.

If we break it down into three pieces (or maybe just two), we have ‘SQL’, ‘ETL’, and ‘Developer’.

SQL is an acronym for Structured Query Language. SQL is a programming language (some may disagree with that) that is designed to help with managing data that is held within databases. Databases use things called objects to house, maintain, and alter data; some object examples are tables, views, and stored procedures. A quick SQL example of how to get all data from a table called Persons would be ‘SELECT * FROM Persons’.

ETL is also an acronym for Extract, Transform, and Load; which is exactly what I do with the data from the SQL portion above. I extract data from tables (or some other source), do some transformation(s) to the data, and then loadit somewhere else. It is worth noting that the extraction source doesn’t always come from database tables; it can be from anything that has data such as a text file, Excel file, or Access database. The transformation phase is where there may be a process to fix/clean the data (such as capitalizing the first letter of names) or using business rules to determine how the data should look like (such as phone numbers should be formatted with no hyphens). Finally, the load stage is the process of moving the data somewhere such as another table or file. Depending on the needs of a project, the current ETL process could be a sub-process of many or a single process of a project.

Developer, of course means I develop.

Software I tend to work with is SQL Server Integration Services (SSIS), SQL Server Management Studio (SSMS), and Microsoft Visual Studio IDE (VS). SSIS is the program where I use tools to develop the ETL process, SSMS is the program I use with SQL to look/mess around with data within databases, and VS is where I handle/maintain database projects.

So there is a high-level view of what a SQL ETL developer (or at least what I do) does.