StarTree — Smart Trigger

Fall 2022 Industry Project

Introduction

Our Client

Our Industry Client, StarTree is a California based startup that provides users/client with Apache Pinot Database Systems as a Service (SaaS) and tools that let them analyze their data and give insights.

StarTree tasked us with building an additional service that they would offer their clients to monitor their data and analyze it in a new way. We had to build a system that would achieve the following:

• Monitor large scale databases and alert users of changes within the data with low latency
• Provide this service in an accessible way where non-technical users can use it without much hassle

The Team

• Kaushik Nambi — Project Manager
• Krishelle Diaz — UX Designer
• Giovanna Briano — UX Designer
• Jason Fu — Developer
• Jonathan Yeung — Developer
• Ria Rajput — Developer
• Rithik Sachdeva — Developer

Timeframe

We were given 6 weeks from from spanning Oct-Nov 2022. This was a tight timeline and we had to work fast and stay flexible at the same time. This led the team to adopting an AGILE workflow where each week we worked in sprints to achieve our goals. Once the project was planned out fully, our timeline looked something like this:

Tools

The main tools we used for this project are split into 3 parts

• Figma for Design and Prototyping
• React to create the frontend
• SpringBoot, Java and MongoDB for the backend

The Project

Basic User Workflow and Inner Workings

The landing page that the user will see when they first enter offers a few things

• It provides statistics on which alerts were triggered, when, and how many times in the stacked bar chart
• It shows us quick statistics about the alerts stored in the form of cards
• There is a table “Alerts” which has all the alerts from where a user can either view or edit them.
• There is a table “Recent Alert Detection Failures” which provides a log of alerts that have been triggered recently and why

The user can click on the blue “Create Alert” button, which will lead them to this page, where they can configure an alert of their choice

• They have the option to give it a name, description, the frequency at which it needs to check for updates, and a dual selection option between a Smart Alert or a Rule Based alert, the former for non-technical users while the latter is for more technically knowledgeable users.

Once the user clicks next, they can configure the notification settings.

• Here they can choose the message they would like to receive when the alert triggers, the people who would receive this alert, and some other miscellaneous settings such as language and sender email.
• The user also has an option to click through an option of alert groups, where the user can add the current alert to a group so that it is organized well in the eyes of the user

Once the alert has been created, it is queried with Apache Pinot and it stores a baseline. Once the baseline has been passed according to the conditions set by the query, then it will send an alert through AWS SES. A sample email could look something like this, where the user configures all these message options

Internally, the data is stored in 3 database collections. The first one is the Alert collection, that stores all the information about the alert

Second is the SmartAlerts cache, that stores the generated alerts for each table. As generating the Alerts is a heavy operation, we cache these alerts to be returned unless there has been a change in the data

Third is the TriggeredStats repository, which stores information about the alerts that were triggered on certain dates, and how many times they were triggered.

Now, you should have a good understanding of some basic workflows and how it works under the hood.

Design

Design Process

The Design team went through the non-traditional path of the Design Process as StarTree provided specific design guidelines and rules that limited and challenged both the research scope and creativity to experiment with different cognitive methods for the design of the interface.

Research

Insights enablers

Analytics & data engineers (Primary persona)

Secondary personas:

• Data platform engineers
• Application Engineers
• Data Scientists

End users: Insights consumers and owners of definitions (Decision makers)

• Business operations team
• Business functions, PMs

Competitive Analysis was conducted to better understand the interface of the software we were creating and to observe how other companies executed their alert triggers for their software.

• We observed and compared the following:

Synthesizing

HMW Affinity Map: The whole team worked together to come up with HMW statements to categorize and prioritize main themes that we wanted to ensure was part of our design and development process. Key things we needed to be mindful of is to use non-jargon terms to avoid confusing the user and to ensure that CTA buttons are apparent.

Ideating:

After synthesizing and thoroughly learning about the product we needed to create, we ideated interactions of how the smart trigger could be executed by the user

Refining the Design

Refining the design had gone through multiple iterations as we had to abide by a very specific Design System with a specific hierarchical style. Challenges we came across were providing clear CTA buttons and ensuring that the landing page was not convoluted.

Development

Architecture Diagram

Backend -

SpringBoot Rest API

• These are some of the endpoints that we have to do things such as create, update etc.

Query and Notification Translator

• It takes the options that the user inputs and constructs the query and the notification object

Learning to use the Pinot API that is provided by StarTree

• StarTree has a Java Client that we can use to interface with a pinot instance

Backend class structure

• This is a rough diagram of how the backend class hierarchy is structured.

Event Scheduler

• This part of the system was responsible to run queries and check if users need to be updated
• Interfaced with an AWS SES service to send emails to users when they need to be notified

Smart Alerts

• This part of the system looked at all the tables in the Pinot cluster and analyzed their data
• The analyzed data was used to suggest useful alerts to the user when they create an alert

Frontend -

Landing page

• Included several components such as a chart and tables with links to other info pages

Create/Edit page

• Included forms in the page for the user to fill out, create and edit are identical in that they share the same form

Info pages

• These included miscellaneous pages, such as viewing all alerts etc.

Takeaways and Challenges

Takeaways

• We learned how to work with APIs that already exist and use them in a new product
• We learned how to organize a large codebase into more manageable and understandable packages

Challenges

• Our team experienced hiccups with some of StarTree’s tools, and it took us some time to get everything working and continue with the project
• We had some confusion about designs that we went back and forth on before coming to a decision. This shifted the timeline for the frontend interface development

Future

The project has been handed off to StarTree, and we expect them to start using it internally in the company to test the feature and then eventually roll it out to their customers

Conclusion

• It was a great experience for everyone on the team to work in a cross functional team where designers and developers had to work together
• Working in such an environment is a learning experience for everyone as it is a good reflection of how it is like to work in an industry

The entire team would like to thank StarTree and their team for providing us with the opportunity to work with them and create this product for them. Their mentorship was a great asset and it was a great learning experience for the entire team. We appreciate everyone at StarTree that contributed to our experience.