With Terminal: Point of View

ยท 3 min read

1. What? Data normalization across various telematics providers, offering a unified API platform to streamline integrations. To simplify the process of connecting disparate systems, ensuring consistent and clean data.

2. Data? is predominantly spatio-temporal (the best kind of data with high precision), meaning it focuses on the "where?" and "when?" aspects. Expect queries primarily on these dimensions alongside other models.

3. How? The core of the product is to allow easy telematics integration into existing applications (for businesses) without the need to worry about integrating with every provider and handling every variant of integration. An ETL pipeline, where the output is normalized models accessed via APIs.

4. And? The way I see it, the core of the solution is "to empower businesses to do what they do best and leave the integration complexities to us." This calls for solidifying the integration with, Top 4: (1) Input and Output connectors, (2) Context and Template Mapping, (3) Predictions and Aggregations, (4) Analytics and Alerts.

Figure 4: Features; Connectors, Mapping, Agrregations and Analytics

4.1. Output connectors: Given the highly frequent and time-series nature of the data, it's less transactional and more analytical. This implies that the use of the data (for businesses) primarily involves storage in analytical databases and BI systems. This is solved with output connectors, making it a one-stop telematics integration solution.

Figure 5: Input and Output Connectors

4.2a. Context Mapping (Enrichments): It's not uncommon to have systems from multiple providers in a vehicle. Context mapping refers to mapping data from different sources. For example, data outside of telematics providers includes monitoring for in-vehicle cargo, electrical/battery systems, tire pressure, engine health, etc., as well as other relevant data such as environmental conditions (weather) and traffic information.

Figure 6: Context Mapping (Enrichments) across Verticals

4.2b. Template Mapping: Although a normalized API makes integrations easy, it often requires further transformations to use it with current systems. Template mapping is a schema transformation layer to define the output schema on top of the normalized models.

Figure 7: Template Mapping (Transformations)

This may seem like an anti-pattern and doesn't necessarily mean it's not normalized anymore. The concept of normalization typically applies to the internal structure of the data and how it's stored. Perform transformations while: Eliminating Redundancy, Ensuring Data Integrity and Optimizing for Queries.

4.3. Predictions and Aggregations: A lot of the data points add no value when they are not aggregated. Examples include speed at a specific moment, a single fuel consumption reading, an isolated engine fault code incident, a one-time tire pressure reading, a brief idle period, a single temperature reading (for engine or cargo), and many more. However, these data points become more usable when aggregated, such as averages, sums, counts, min/max values, percentiles, medians, etc.

On the same lines, another use case is down-sampling with aggregations, which ensures that the quality of data is not lost while reducing the number of entries/events.

Figure 8: Stateful Stream Processing

Boils down to: Windowing techniques to have context of prior data, which also evolves to predictions (and anomalies). One good example is MindsDB: to enrich data with AI-generated content.

Predictions and Aggregations based on prior data is now a new data source for context mapping (Section 4.2a).

4.4. Analytics and Alerts: Dotted lines in (Figure 4) for a reason! Heading towards analytics and alerts is stepping on the provider's/business's arena, but Analytics/Database as a Service (while still being API-first) is vast enough to coexist.

Figure 9: Batch, Stream and Direct Query Analytics

In fact, aggregations from section 4.3 are a sub-section of analytics, emphasizing "Streaming Analytics" alongside batch analytics and direct queries on the database.

5. What's next? The potential and possibilities are endless.

Tangentials: UI widgets - in popular front-end frameworks to add on to existing applications (Stripe model), Generative AI - on telematics data across providers for drivers, fleet managers, insurance providers, etc., Time and Location Analytics, and many more.

But I'm sure someone back there said, "this isn't our focus," when going over the above "Top 4" feature/enhancement suggestions. That's where I conclude; to know more about what's next?

6. Relevant blog posts: delving into the details of Telemetry, CDC, Spatial Indexes, Anomaly Detection, and Serverless Architecture.

Cite this article as: Adesh Nalpet Adimurthy. (Jun 13, 2024). With Terminal: Point of View. PyBlog. https://www.pyblog.xyz/notes/with-terminal-pov

#index table of contents