Infrastructure: How it fits together
What you see
The Data Viz Starter Pack is built to be a no-code environment that empowers your data analyst to focus on building reports and impacting their organization. The Starter Pack has a number of data sets that are enabled via the console below and may be used in Google Data Studio. By default the Starter Pack also comes with a report library to help your school get started quickly.
Behind the scenes
The core of the Starter Pack is a open source scripting library written in Python and SQL. When a school implements the Starter Pack they receive a forked copy of this library. This makes your life easier in a few ways. First, you have a copy that is specific to your organization! This is important because small tweaks can now be made to ensure the data sets truly meet your needs. Calculate a metric differently than we do? Not a problem! In addition to that, the forked copy is tied to the main code repository. This means that as more integrations are added such as a new edTech application, you can pull in this new code!
The scripting library is run on a virtual machine in your Google Apps Cloud environment. This is super cool because it means your data does not leave your organization. You have a small virtual server that is refreshing data sets with you in complete control. The scripting library has a small Flask web app that powers the console you see at the top of this post.
The scripting library will send your data to one of two places. If you are a small organization, you should be OK using Google Sheets as your data store. This has the advantage of having the raw data sets seen in an approachable form. Want to see what your Student Demographics & Enrollment data set looks like? Just open the Google Sheet. If you have a large amount of data, the library can just as easily store your data in a Google Cloud SQL database located in the same Google Cloud project as your virtual machine.
As the scripts run they keep detailed logs as to what is going on. This is saved to a console where you can see all logging in one place. Over time the Starter Pack will email out alerts if it ever runs into any issues.
That’s it! That is a little glance behind the curtain to understand how the various components of the Data Viz Starter Pack come together. In future posts we will explain how those data sets are then tied into the reporting tool that we chose, Google Data Studio.