Analytics teams are positioned to guide everyone in a company to make better decisions. But how well are companies measuring the performance of their analytics teams? What types of questions are they dealing with, and how do their actions impact the company’s growth trajectory?
We dive into these topics and much more in the latest episode of our podcast. Our CTO Drew Beaupre chats with Osman Ghandour, Co-Founder of Soal Data. Osman explains that ad hoc analytics refers to unplanned, reactive work that data and analytics teams often have to handle. While this type of work can easily be seen as a distraction, it can also be an opportunity for these teams to ease decision-making throughout a company. As Osman points out, there’s a fine line between resolving a one-time ask vs recognizing the types of questions that come up over and over again. Soal Data’s mission is to shine a light on all types of analytics questions, to show which ad hoc requests should be given more close attention. Drew clarifies their value this way: if analytics and data teams receive repeat ad hoc requests of a similar nature, there’s a much higher likelihood that those requests point to valuable areas for the business to focus on. Soal Data helps companies identify these patterns automatically. Think of their solution as analytics for your analytics.
Data and analytics teams can be structured in three distinct ways to address ad hoc requests:
- No special structure. A company’s existing analytics and data teams must drop whatever they’re doing whenever an ad hoc request comes in. In this scenario, the same people must manage proactive and reactive work at the same time. While this scenario is the most common, it’s also the most likely to increase context switching, burnout, and employee turnover.
- A hybrid approach rotates the responsibility for addressing ad hoc requests among different members of the analytics and data teams on a weekly basis. This scenario reduces context switching and its associated burnout, and it has the added benefit of giving everyone on the team a broader perspective of what the team is working on.
- The fully dedicated ad hoc team. Osman often sees this setup in certain industries that receive a non-stop flow of questions, especially in insurance, finance, or anything related to compliance.
Osman highlights the importance of measuring the value of ad hoc work, since today’s ad hoc requests might very easily become part of a company’s product roadmap tomorrow. Soal Data allows companies to identify that tipping point quickly and efficiently.
Transcription
Drew Beaupre (00:05):
Hi everyone. My name is Drew Bore. I'm the CTO of Mammoth Growth, and today on the podcast I have Osman Ghandour, the co-founder and CEO of Soal. Welcome to the podcast and tell me a bit about yourself and what do you do.
Osman Ghandour (00:19):
Hey Drew, thank you so much for having me on as a guest. I really, really appreciate it. So I run a company in early stage startup in the data space called Soal. We build a platform basically for analytics teams to do ad hoc work and what ad hoc means, and we'll get into this later, but anything that is unplanned. And so all those requests that your data team gets, we help your team handle that at ease.
DB (00:48):
We first spoke a while back, you introduced me to this idea of ad hoc analytics and I don't know up to that point, I just always thought it as with something I would call a fire drill. And you showed me kind of a different lens. I have a bit of an aha moment. How do you view ad hoc analytics just as a concept and why is it not just a series of fire drills?
OG (01:14):
So the way I like to view ad hoc analytics is as being everything that is unplanned, all the work that is reactive as opposed to proactive. And so what does that actually mean? At the beginning of the quarter, the head of data will come and say, or the head of analytics will come and say, “Hey, these are the projects we need to work on,” and that looks very different than the actual day-to-day of what the team is working. They'll be pulled into different tasks, different requests, in and out all the time. And all of that work is kind of unorganized and inefficient and unplanned. That is what I refer to as ad hoc work, as opposed to being like, yeah, we're going to build out these specific data products and we knew that three months ago that we're going to do this. That is kind of more planned, proactive work.
DB (02:02):
And so it's not inherently a bad thing. And so how we do it at Mammoth, most of the work we do is very proactive. We meticulously plan our work even if we're working on one to two week sprints, but then there's always a project that kind of feels a bit reactive. Are those two things compatible with each other or is it best served with a complementary team that either does one or the other
OG (02:36):
So they can be put into the same team or they can be separate functions? And I'll talk a little bit about the structure and how that can be done in a sec, but I want to talk a little bit about your first point, which is around initially viewing it as maybe a bad thing, right? As being like, oh, this is something else that my team has to take care of. We hadn't originally thought of this, but someone's coming in asking for it. One key insight that we had is that analytics teams are there to support decision-making throughout the rest of the company. And when a stakeholder comes and says, “Hey, analytics team, I need this and this and this, I need help with this decision.” We want to be there for them. As an analytics team, our job is to support decision making. So it really kind of pulls the company in a non-data driven direction to say, “Hey, stakeholder, you know what? We're not going to support you on this ask that you're coming for. Just come look at a dashboard. We'll have something relevant in three months.” In some cases, that's fine and that's necessary, but in many other cases, this serves as an opportunity for the analytics team to get closer to the business teams and vice versa. And it serves as an opportunity to actually influence the decision making in a data-driven way throughout the company. So yeah,
DB (04:02):
I do a lot of ad hoc analytics internally on our own internal data platform. And to be honest, it's actually a really fun experience when you get asked a question and you can leverage the data assets that you've built in a really rapid fashion where you don't have to go and build a whole bunch of infrastructure, you can actually use exactly what you've done and it's actually kind of nice knowing that you've done it the right way because you're able to answer this one-off question in a really rapid way and you get to move on and now you don't have to actually support that infrastructure going forward versus if it came through, oh, okay, it's got to go through the pipeline. We're going to build a dashboard for you and you're only going to use it once, but we still have to maintain that dashboard for eternity,
OG (04:46):
Right? Yeah, there's a fine line between bringing something to fruition for a one-time ask versus knowing the types of questions that are asked over and over again and having those data products that are ready for that. And that kind of brings me to a point about how they blend together, how those one-off asks blend into those longer term projects. And so things inevitably start as a one-off ask, but then those one-off asks become repetitive. And that's a big kind of gap in the industry today, right? It's me. Let's say I'm an analytics leader. I'm a VP of analytics at a Fortune 500 company and I have this big, nice strong analytics team. I don't really know what my team is dealing with on a day-to-day basis in terms of requests and asks to the degree to which I can say, these are the things that we need to be focusing on because our stakeholders are engaging with this ad hoc work that our team does very heavily, right? Engagement is a whole issue today. There's very little measure of engagement. I'm coming to you and you're the analytics team, I'm a business stakeholder, and I ask for something and you give it to me. You really don't hear from me again unless I have follow-up questions. You don't know how many times I referenced that number, that analysis. You don't know how many times I used it, if I used it, if I really didn't use it. And so yeah, it's trading that closeness, that collaborative relationship is extremely important
DB (06:26):
Actually. You could argue that that is a much stronger signal of what's needed, but it's actually truly needed in the business. Then if you came at it just from a traditional planned BI initiative, if you're getting repeat requests of a similar nature over and over and over again, it's a much higher probability that is a really useful business thing to formalize and now prioritize to create something that's more self-service and repeatable.
OG (07:02):
And so think of it as starting a business or listening to the market. The market speaks the truth. And so in this case, we're speaking internally, my stakeholders are my customers, and so I want to be listening to what they need and their feedback rather than pushing them away and saying, I'm going to build something. And you can use it rather think, Hey, what do you need? And I agree a hundred percent with what you're saying. Yeah, makes sense.
DB (07:28):
How do you see, see these teams, the makeup of these kinds of teams?
OG (07:33):
And so there are a few different, so again, I've spoken to a ton of heads of data, VPs of analytics. There are many different ways that ad hoc work can be done. Most there are, let's say, let's split it up into three different kind of categories. Category one, which is unfortunately the most common, is saying, “Hey, you know what? We're going to have our team of X number of people and they're going to get questions from time to time - or really all the time - and they're going to have to work on those questions, and they're just going to have to stop whatever they're currently doing and work on those questions.” It could be according to some priority or according to some task assignment, that's fine, but it's the same people doing proactive and reactive work at the same time. That's one that's
DB (08:18):
The classic fire drill where it just fills up all your time.
OG (08:24):
And this is something that analysts really hate because these requests can be distracting. They can be really a suck on their personal productivity saying, “Hey, my manager's holding me accountable to X, Y, and Z. I don't need to be working on A, B and C, even though A, B and C might be really valuable for the stakeholders.” And so it actually leads to more analyst burnout, more turnover and doesn't make anyone happy. Other kind of, let's call it like a hybrid approach, is having people on call saying, “Hey, this week we're going to have Tom and Jerry on call and they're going to be handling requests that come in.” There are a couple of nice aspects to this. One really nice one is that Tom and Jerry then get during that week, they have a wider view of what the data team's work is, and they know a little bit about everything rather than being completely into their own holes and domains.
(09:27):
The next week is not going to be Tom and Jerry, it's going to be someone else and someone else. And so this rotational structure is another common one, and they're kind of trying to act like IT teams, which have this kind of on-call structure, a very built in the last category is the fully dedicated ad hoc team. And so that doesn't arise in every industry or in every sector. This arises in cases where there are certain segments of the business that get a ton of questions. And so things that come to mind are insurance companies, anything to do with compliance. So within the financial services sector as well, there are dedicated teams who are getting an onslaught of questions, and so they need those analysts to be working on those constantly. And so that's how I would split things up.
DB (10:22):
The on-call idea is quite interesting. There's the benefit that it rotates people closer to ultimately their customers. Like you said, your stakeholders, the people who are needing these analyses, they're your customers in a way. And next you potentially closer to those customers. And so then when you come back into the planned rotation, you have a little bit more of an insight as to the models that you're building and the act of work that you're doing, how it ultimately will be used out in the wild.
OG (11:04):
Yeah, that's exactly what we've seen. In addition, like I said before, to reducing the context switching that is associated with having no kind of structure around ad hoc work, that exposure to the business side of things is extremely valuable.
DB (11:27):
What kind of functions, you mentioned head of data. What kind of functions do these teams typically roll up into?
OG (11:34):
So sometimes there are two general cases, and I'm sure you guys are very aware of this, is how analytics and data split up in the company. There's either a central data team that could be data engineering plus analytics, or maybe they just have a central data engineering team and then distributed analytics team, right? Marketing has its own analytics team. Finance has its own analytics team. So those are the two. Data engineering is very rarely distributed from what we've seen. And so in the case that it is, is only in extremely large companies. So there's some elements of decentralization, and if you're in a more decentralized structure where your analytics is decentralized, then that'll be rolling up into, Hey, this analytics team belongs to marketing. That one belongs to operations, that one belongs to finance. And so that's one case again, yeah, other case, it's your VP of analytics, director of analytics, head of data for the fully centralized structure,
DB (12:39):
Does it have to be data warehouse teams? Can it be product analytics like getting into tools like Amplitude, Mixpanel, and such?
OG (12:48):
Definitely. Definitely. Yeah, that's a whole nother thing, and I'm glad you brought it up. There are these teams that have a little bit of a wider scope, and again, product analytics is a very good example of that. And we do see some of the cases of those teams being a little bit of a catchall for everything, and especially product analytics. They do get a ton of questions across the board because everything relates to product usage, to all types of stuff.
DB (13:22):
So from we're talking about, it sounds like there's a tactical and a strategic element, and the work that you're doing in ad hoc analytics can actually lead to initiatives in more planned work. Is there kind of a natural flow to one leading to the other?
OG (13:51):
So the natural flow today is based on intuition, right? It's saying, “I'm a manager, I'm an analytics manager, and hey, my team's getting these requests all the time, and maybe I'm grooming those requests, maybe I'm assigning those requests to my team. I'm like, Hey, I've seen this before. I saw this a few weeks ago. Why don't we kind of prioritize this on our roadmap?” That's really just kind of intuition based. And sometimes those requests will all be in one place, let's say in a ticketing system like Jira or in somewhere else, and you can actually look back and say, oh, this question has been asked five different ways over the past couple of weeks. And so that's kind of how it usually happens. The missing part today, and I think where the industry should be going is not just looking at the questions that are being asked, but the engagement with the work that the data team is doing. And many BI tools give usage dashboard usage, but being able to tie that dashboard usage to a particular question that was asked, for example, could be very, very valuable. So looking on more on the engagement side I think is where things should be going.
DB (15:08):
So that's something that we're finding a lot in the industry in the last six months or so, is that teams have to demonstrate their value and justify their budget a lot more in organizations these days. So how can an ad hoc team sort of measure that value and sort of communicate the overall performance and value that they're bringing to the company?
OG (15:37):
It's extremely difficult as for data teams and analytics teams to measure by what they bring to the business, which is undoubtedly in many cases, very, very high today, how does value assessment happen? I mean, unless there was a particular case where, hey, we closed this customer because this sales rep had this information. In those cases, it's a little clear the value, but other than that, in terms of day-to-day operations, the value is really not being quantified. I would say it's just based on the stakeholder intuition as to whether they're being supported by the data team, which actually adds to the importance of making sure those stakeholders have the information they need to make the decisions they need to make. We should go, I think, as an industry into the direction of saying, “Hey, this is the value of the work that we're bringing.” And the core issue with that is that, and the value of an insight depends on what will be done with that insight, and that is by nature beyond the scope or the knowledge of the data analytics team. And so you'll need some type of input from those stakeholders based on, Hey, what are you doing with this piece of information? Can you put an approximate dollar value on what you would pay to get this piece of information right?
(17:14):
There is going to have to be more collaboration with those stakeholder teams in order to come to some value conclusion.
DB (17:22):
It sounds a lot like customer experience and the types of feedback that are needed in order to evaluate the performance of a CX team.
OG (17:35):
I think that's a great analogy. A hundred percent.
DB (17:40):
So what are, I mentioned that I'm often running my own ad hoc queries. I'm just into Snowflake UI. I've always, how could I be more efficient or more repeatable? A lot of times I'm in the Snowflake UI, I'll write some SQL, but it's kind of ephemeral and I might come back weeks later and I got to rewrite it over and over again. So what kind of tools or new developments are you experiencing in the ad hoc world?
OG (18:16):
Yeah, so aside from every team having, like you said, their cloud data warehouse, their BI tool, their messaging tool, their ticketing tool, your example that you gave is, it's a very common one. Well, this query that I spent all this time working on, I don't want to have to do it again every single time. And that's not even to say that someone else should be writing it. Even I may have written it just because written it doesn't mean that he should write it from scratch too. He should just take what I need. And since today, if it's just sitting in some local file or even some shared random file in some cloud environment, that person's not going to know that you did that work. You will only know if you remember that you did that work. So I've seen cases where people are copy and pasting SQL code into Jira, into Slack, into God knows where. And obviously things can get stale, things can get lost, not very searchable.
DB (19:16):
I've seen a giant Google Doc just people just keep adding SQL statements to those Google Doc of like a recipe book.
OG (19:21):
Yeah, yeah. Those kind of people like to have some central repo of where their work sits, and obviously that's not the best way to do it. Yeah.
DB (19:35):
So tell me about Soal and how it plays into all these ideas.
OG (19:39):
Yeah. So is a platform that actually integrates with your cloud data warehouse, your BI tool, your ticketing system, your messaging system in order to create an extremely streamlined workflow for ad hoc work. And it gives you a place to quantify the value of your ad hoc work. And so for us, we have two main objectives. The long and short of it is that we want to reduce the time it takes for an ad hoc analysis or for a question to be resolved. That's one. And two, we want to give data teams the tools to quantify the impact and demonstrate the value of their work and of their ad hoc work, which today is completely ignored at the end of the quarter. When you look at what your team has done, you probably only focus on those planned projects and you don't really take into account all of the other value that your team brought to the company. And so
DB (20:38):
What's the experience of, let's say I've got a question, what's my experience going to be in order to order to get my answer?
OG (20:49):
So any business stakeholder or anyone at all can ask a question from their messaging platform. So whether that's Slack or teams. From there, we basically create a workspace for that request in our platform. And in that workspace, you can think of having a data notebook to work on that request. So being able to run queries because we've connected to your cloud warehouse, being able to bring in references to dashboards that you've built, being able to explain answers, that kind of centralization component is really important to us. After that, you have a recommendation engine that actually feeds you the resources you need to do your work faster. So, “Hey, Drew, this request that you're working on, John worked on something very similar three months ago.” You should think about reusing these bits of his work cuts down on this is what really cuts down on that discovery time needed. And then after that,
DB (21:44):
We're not reinventing the wheel every single time.
OG (21:47):
Precisely, precisely. Then after that, we follow up and we say, “Hey, stakeholder, what do you think? Here's the link to the response. Where are you happy with it?” They didn't meet your needs. And then we track their engagement. Did they go in and view the answer once for 10 seconds or was it passed on to the team? And I had a hundred views over the past two weeks, and so it was maybe something that we should pay some more attention to.
DB (22:15):
So as the original asker, I'm interacting with it inside of the notebook itself. I'm still, so that's my sort of portal into it. I'm not just taking the CSV or the Excel sheet and running away with it.
OG (22:33):
So you as the analyst is kind of as a dedicated workspace where they can execute their work in the platform for that request that is completely separated from the work done for other requests. And then the portal for the business stakeholder is basically Slack or Teams, except for viewing the actual work itself. They can access the platform from Slack or Teams seamlessly.
DB (23:00):
What's the experience from the lens of the analyst in terms of context switching, for example, like handling multiple concurrent requests?
OG (23:12):
And so the nice thing is instead of splitting things up across a bunch of different files, a bunch of different tools, you have one data notebook, one page basically for each request. And so across the team you can be like, “Hey Drew, I need you to work on this request that I'm kind of halfway through, but I don't have time to finish.” If you go there, you can see all the work that was done for that request in one place. You can see the intake information, you can see all context, all comments, all follow ups, all the work that was done itself, all references to work all in that one place. So that's kind of how it looks from an analyst's point of view.
DB (23:54):
What happens three weeks later when the original asker has a follow on question
OG (24:01):
So you can link questions to other questions. And so when you see a question like the notebook for a question, you can see the question that it has been linked to if a link has been made. So you can not only refer to a dashboard that you've made, but you can refer to a previous question that's been asked. And the answer for that question,
DB (24:20):
Who? So who's using Soal?
OG (24:24):
Yeah. So medium to large sized companies who have analytics teams who have trouble with those two things that I said at the beginning. They have trouble with a lot of repeated work. They have trouble with having a searchable repo of their data work. That's one thing. And teams who really have no idea how to prove to the higher ups the value that their team brings, really heads of data need, what we call analytics for analytics, all that, that they're going to have to talk to their peers, talk to the executives about how their team is performing, what their team's doing at a very granular level.
DB (25:10):
Yeah, very much like CX. So how does the repeatable work where, how do you see the repeatable work flowing into the planned pipeline, like the backlog and roadmap?
OG (25:27):
So what we do is we extract the topics from the questions and the subjects. And so we can tell you if certain data sets or certain topics are getting a lot of engagement or a lot of questions, a lot of interest basically. And that is the starting point for taking something that is ad hoc and turning it in a very data-driven way into a longer term data asset.
DB (25:53):
It could be done in sort of a backlog grooming session. It's like, okay, what's the top things in this platform now? Let's go and prioritize it in our planned work.
OG (26:04):
Precisely. Yeah, that's exactly how it works.
DB (26:08):
Amazing. Well, thank you very much for your time. I'm really excited to get back into running some ad hoc sql.
OG (26:19):
Thank you for having me. It is been a pleasure.
DB (26:22):
Thank you.
