Data Story — How to Get Detailed Data Requirement Right

Business Intention — Understand it Deeply

I like to call it intention because sometime how we achieve it could be different from what is requested to us. The intention is what is important. Being creative to achieve the intention might mean a lot less work to do meanwhile maximizing the impact. However, this require understanding of business context. You might want to spend some story mapping sessions to understand the intention. A good reference for this is User Story Mapping book by Jeff Patton and Peter Economy.

Data Schema — Elaborate the Definition

In the end, when you are talking about data requirement, it’s all about schema. What data is required, and what are the definitions. A good schema at least has column name, description, data type, and sample value. It is very important to get data description right, elaborate it, because here is misunderstanding usually happen. You might also want to understand what are the unique identifier of data expected, the “primary keys” to understand how granular the data should be.

Realtime or Batch — Don’t Go Realtime Unless you Really, Really Need to

For the sake of definition, let say realtime means less than 10 seconds (it’s not system-critical-realtime), and batch is usually scheduled (hourly, daily, etc). If your user request for something realtime, you really have to dig deeper whether it really has to be realtime. Often, it’s just something that’s nice to have.

Data Freshness — Your Processing Time Budget

This detail usually forgotten to be discussed when you decide to go with batch. For example, your user need the data to be ready at 8AM. But until when? Is it until the end of the previous day (ie. 23.59 yesterday), or is it 7.59AM today?

The Interface — API, SQL, or Semantic Model

This one is simple. How you will serve the data. How your user will retrieve the data. API usually being called by machine. SQL usually written by analyst or scientist. Semantic model usually made for business user on BI tools for data democratization.

After all, it’s just a communication tool…

Those detail above is just a guideline. You might want to add a lot more things that suit your organization and use case better. There’s no wrong or right for those requirement detail. You do it right when you and your user have shared understanding on what you want to achieve.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rendy B. Junior

Rendy B. Junior

Crazy dad. Data technology enthusiast. Youtube: Insinyur Data