Architecture Overview

The Use Cases for metacog™ platform can be divided in two high level scenarios: Data Collecting and Data Extraction and Analysis. In order to support these scenarios the metacog platform is composed of several components at both the client and server sides. This section presents a general overview of each of them.

Data Collection

In order to support the traffic of hundreds and thousands of students sending events in an scalable and fault-tolerant way, the platform makes use of a client-side component known as the metacog logger that communicates with the Instrumentation Input API.

metacog logger

To use the metacog platform, the first step is to integrate the metacog logger to your learning objects. This is done at the source-code level and involves the matching of function calls and variables that may be relevant for logging to the platform.

This process is called "instrumentation". When a learning object has the metacog logger properly integrated, we say it is an "instrumented learning object."

To learn more about the instrumentation process, check the Logger Instrumentation Guide.

Instrumentation Input API

The real-time traffic of all the students using their learning objects is driven through the instrumentation-input API, a highly available and fault-tolerant service.

The logger and the instrumentation-input API work together to guarantee that all the events sent by the client are received on the server side. Even in the scenario of a temporary network failure, the logger keeps a queue of pending messages and delivers them once the connection is reestablished.

Validation

Events that pass through the instrumentation-input API are validated and authenticated. Authentication happens at the organizational level, not at the student level, and metacog does not ask for any of your students’ personal or sensitive information to be included in the logged messages. (See more in the FAQ.)

Storage

The validated events are stored in the main database, without any restrictions on the amount or age of the records.

Data Processing and Analysis

Once all the data is stored in the main database, it can be retrieved from, and processed by different services: from raw data extraction to complex Educational Data Mining processes and visualizations tools, to API's, metacog offers you a wide range of options for extracting meaningful information from your data.

Extraction and Packaging

Every time a request is received to retrieve stored events, the extraction service is executed, recovering the relevant records based on the query parameters, and packaging them to be returned to the client.

These tasks are executed as a background process, so the client must poll or wait for notification of termination of the tasks through the metacog API. (See below.)

Retrieval-Data API

The retrieval-data API is a JSON-based REST API that allows users of the metacog platform to request data originated in learning objects instrumented using the instrumentation input API.

Given the large amount of data logged by each learning object, the results of a query are not returned immediately. Instead, a data-request object is created in the server to keep track of the current request, based on the filter parameters defined by the client, while the real processing is executed as a background process.

The API allows polling the data-request object in order to know the status of the background process. The client may also wait for an automatic e-mail notification to a prevalidated e-mail address, indicating that the process is finished, and the data can then be downloaded.

Training and Playback Toolbar

While the former Use Case of collecting data from an instrumented learning object is covered by the use of the Logger component in client side, the metacog client library also includes a robust set of UI components with powerful features for analytic tasks: the ability of recording sample sessions to be used as training sets for machine-learning algorithms, and the ability of playing-back recorded sessions, all of these without abandoning the UI of the instrumented learning object.

In order to know more about the Training and Playback Toolbar, please check the Playback Toolbar Integration Guide.

Check the Client Library documentation to learn more about the available services.