Data Flow Specification

Overview

This document outlines the data flow processes for a system designed to handle GPS data, generate headcount and manhour snapshots, and provide data to operational and analytical tools. The system integrates multiple components, including Traccar, Georest API, CouchDB, and TrekkerTree, to achieve its functionality.

Data Flow Components

Trekker System Architecture

1. GPS Devices

  • Source: GPS-enabled devices (e.g., smartphones, trackers).
  • Functionality:
  • Collect real-time GPS data.
  • Transmit location information to the system.
  • Output: GPS data.

2. Traccar (Eg: Port 5051)

  • Purpose:
  • Ingest GPS data from multiple GPS devices.
  • Process and store raw GPS data.
  • Data Received: GPS data from devices.
  • Data Output:
  • Raw GPS records sent to the Georest API.
  • Other API integrations for Geofence and Device management.

3. Georest API

  • Purpose:
  • Act as an intermediary between Traccar and other system components.
  • Replicate GPS records for further processing.
  • Data Input: Raw GPS records from Traccar.
  • Data Output:
  • Raw GPS data passed to snapshot generation processes.

4. Headcount Snapshot Generation

  • Purpose:
  • Generate headcount snapshots at regular intervals (every 5 minutes).
  • Data Input: Raw GPS records from Georest API.
  • Data Output:
  • Headcount snapshot data sent to CouchDB for storage.
  • Shared with TrekkerTree for analytical visualization.

5. Manhour Snapshot Generation

  • Purpose:
  • Generate summarized manhour data every 6 hours.
  • Data Input: Processed data from CouchDB.
  • Data Output:
  • Manhour snapshot data stored in CouchDB.

6. CouchDB

  • Purpose:
  • Serve as the central database for storing processed data.
  • Enable fast access to snapshot data for operations and analytics.
  • Data Stored:
  • Headcount snapshots.
  • Manhour snapshots.
  • Users, workzones, and workplans.
  • Data Output:
  • Provides data to WebUI for operations.
  • Supplies data to TrekkerTree for analytical display.

7. TrekkerTree

  • Purpose:
  • Analyze and visualize headcount snapshots.
  • Display operational insights through dashboards.
  • Data Input:
  • Headcount snapshots from CouchDB.
  • Data Output:
  • Analytical visualizations delivered to end-users.

8. WebUI

  • Purpose:
  • Handle user operations, such as registration and device management.
  • Display headcount and manhour data for operational use.
  • Data Input: Snapshot data from CouchDB.
  • Data Output:
  • Operational tools for user and device management.

Data Flow Summary

Processes

  1. Data Ingestion:
  2. GPS devices send data to Traccar on port 5051.
  3. Data Replication:
  4. Traccar replicates raw GPS records to the Georest API.
  5. Snapshot Generation:
  6. Georest processes raw GPS data to generate:
    • Headcount snapshots every 5 minutes.
    • Manhour snapshots every 6 hours.
  7. Storage and Access:
  8. CouchDB stores snapshot data and metadata.
  9. WebUI and TrekkerTree access data for operational and analytical use.

Data Flow Specification

Input Data

  1. GPS Data:
  2. Latitude, longitude, timestamp.

  3. Sent by GPS devices to Traccar.

  4. User Data:

  5. Workplans, workzones, and user profiles.
  6. Input through WebUI.

Processed Data

  1. Headcount Snapshots:
  2. Summarizes GPS data into user headcounts.

  3. Generated every 5 minutes.

  4. Manhour Snapshots:

  5. Aggregates user activity data over 6-hour intervals.

Output Data

  1. Analytical Data:
  2. Delivered to TrekkerTree for visualization.
  3. Operational Data:
  4. Delivered to WebUI for device management and operational reporting.

Key Notes

  • Update Intervals:
  • Headcount snapshots are updated every 5 minutes.
  • Manhour snapshots are summarized every 6 hours.
  • Integration Points:
  • Georest API integrates raw GPS data with snapshot generation processes.
  • CouchDB centralizes data storage for WebUI and TrekkerTree.

Let me know if additional details or refinements are needed!