diff --git a/docs/_summary.md b/docs/_summary.md index baa49371709..dee8a31984f 100644 --- a/docs/_summary.md +++ b/docs/_summary.md @@ -6,12 +6,6 @@ * [Testing and Debugging](newbs_testing_debugging.md) * [Getting Help](getting_started_getting_help.md) -* Breaking Changes - * [Overview](breaking_changes.md) - * [My Pull Request Was Flagged](breaking_changes_instructions.md) - * History - * [2019 Aug 30](ChangeLog/20190830.md) - * FAQs * [General FAQ](faq_general.md) * [Build/Compile QMK](faq_build.md) @@ -19,8 +13,14 @@ * [Keymap](faq_keymap.md) * [Driver Installation with Zadig](driver_installation_zadig.md) +* Configurator + * [Overview](newbs_building_firmware_configurator.md) + * [Keyboard Support](reference_configurator_support.md) + * QMK API + * [Overview](api_overview.md) + * [API Documentation](api_docs.md) + * Using QMK - * [Support](getting_started_getting_help.md) * Guides * [ARM Debugging Guide](arm_debugging.md) * [Best Git Practices](newbs_git_best_practices.md) @@ -89,8 +89,13 @@ * [Using Eclipse with QMK](other_eclipse.md) * [Using VSCode with QMK](other_vscode.md) - * Developing QMK + * Breaking Changes + * [Overview](breaking_changes.md) + * [My Pull Request Was Flagged](breaking_changes_instructions.md) + * History + * [2019 Aug 30](ChangeLog/20190830.md) + * QMK Reference * [Translating the QMK Docs](translating.md) * [Config Options](config_options.md) @@ -101,7 +106,6 @@ * [Community Layouts](feature_layouts.md) * [Unit Testing](unit_testing.md) * [Useful Functions](ref_functions.md) - * [Configurator Support](reference_configurator_support.md) * [info.json Format](reference_info_json.md) * C Development @@ -121,6 +125,11 @@ * [QMK CLI Config](cli_configuration.md) * [Python CLI Development](cli_development.md) + * Configurator Development + * QMK API + * [Development Environment](api_development_environment.md) + * [Architecture Overview](api_development_overview.md) + * For a Deeper Understanding * [How Keyboards Work](how_keyboards_work.md) * [Understanding QMK](understanding_qmk.md) diff --git a/docs/api_development_environment.md b/docs/api_development_environment.md new file mode 100644 index 00000000000..50647c42995 --- /dev/null +++ b/docs/api_development_environment.md @@ -0,0 +1,3 @@ +# Development Environment Setup + +To setup a development stack head over to the [qmk_web_stack](https://github.com/qmk/qmk_web_stack). diff --git a/docs/api_development_overview.md b/docs/api_development_overview.md new file mode 100644 index 00000000000..e55d0341006 --- /dev/null +++ b/docs/api_development_overview.md @@ -0,0 +1,44 @@ +# QMK Compiler Development Guide + +This page attempts to introduce developers to the QMK Compiler. It does not go into nitty gritty details- for that you should read code. What this will give you is a framework to hang your understanding on as you read the code. + +# Overview + +The QMK Compile API consists of a few movings parts: + +![Architecture Diagram](https://raw.githubusercontent.com/qmk/qmk_api/master/docs/architecture.svg) + +API Clients interact exclusively with the API service. This is where they submit jobs, check status, and download results. The API service inserts compile jobs into [Redis Queue](https://python-rq.org) and checks both RQ and S3 for the results of those jobs. + +Workers fetch new compile jobs from RQ, compile them, and then upload the source and the binary to an S3 compatible storage engine. + +# Workers + +QMK Compiler Workers are responsible for doing the actual building. When a worker pulls a job from RQ it does several things to complete that job: + +* Make a fresh qmk_firmware checkout +* Use the supplied layers and keyboard metadata to build a `keymap.c` +* Build the firmware +* Zip a copy of the source +* Upload the firmware, source zip, and a metadata file to S3. +* Report the status of the job to RQ + +# API Service + +The API service is a relatively simple Flask application. There are a few main views you should understand. + +## @app.route('/v1/compile', methods=['POST']) + +This is the main entrypoint for the API. A client's interaction starts here. The client POST's a JSON document describing their keyboard, and the API does some (very) basic validation of that JSON before submitting the compile job. + +## @app.route('/v1/compile/<string:job_id>', methods=['GET']) + +This is the most frequently called endpoint. It pulls the job details from redis, if they're still available, or the cached job details on S3 if they're not. + +## @app.route('/v1/compile/<string:job_id>/download', methods=['GET']) + +This method allows users to download the compiled firmware file. + +## @app.route('/v1/compile/<string:job_id>/source', methods=['GET']) + +This method allows users to download the source for their firmware. diff --git a/docs/api_docs.md b/docs/api_docs.md new file mode 100644 index 00000000000..28a7dd71da1 --- /dev/null +++ b/docs/api_docs.md @@ -0,0 +1,68 @@ +# QMK API + +This page describes using the QMK API. If you are an application developer you can use this API to compile firmware for any [QMK](https://qmk.fm) Keyboard. + +## Overview + +This service is an asynchronous API for compiling custom keymaps. You POST some JSON to the API, periodically check the status, and when your firmware has finished compiling you can download the resulting firmware and (if desired) source code for that firmware. + +#### Example JSON Payload: + +```json +{ + "keyboard": "clueboard/66/rev2", + "keymap": "my_awesome_keymap", + "layout": "LAYOUT_all", + "layers": [ + ["KC_GRV","KC_1","KC_2","KC_3","KC_4","KC_5","KC_6","KC_7","KC_8","KC_9","KC_0","KC_MINS","KC_EQL","KC_GRV","KC_BSPC","KC_PGUP","KC_TAB","KC_Q","KC_W","KC_E","KC_R","KC_T","KC_Y","KC_U","KC_I","KC_O","KC_P","KC_LBRC","KC_RBRC","KC_BSLS","KC_PGDN","KC_CAPS","KC_A","KC_S","KC_D","KC_F","KC_G","KC_H","KC_J","KC_K","KC_L","KC_SCLN","KC_QUOT","KC_NUHS","KC_ENT","KC_LSFT","KC_NUBS","KC_Z","KC_X","KC_C","KC_V","KC_B","KC_N","KC_M","KC_COMM","KC_DOT","KC_SLSH","KC_RO","KC_RSFT","KC_UP","KC_LCTL","KC_LGUI","KC_LALT","KC_MHEN","KC_SPC","KC_SPC","KC_HENK","KC_RALT","KC_RCTL","MO(1)","KC_LEFT","KC_DOWN","KC_RIGHT"], + ["KC_ESC","KC_F1","KC_F2","KC_F3","KC_F4","KC_F5","KC_F6","KC_F7","KC_F8","KC_F9","KC_F10","KC_F11","KC_F12","KC_TRNS","KC_DEL","BL_STEP","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","_______","KC_TRNS","KC_PSCR","KC_SLCK","KC_PAUS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","MO(2)","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_PGUP","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","MO(1)","KC_LEFT","KC_PGDN","KC_RGHT"], + ["KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","RESET","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","MO(2)","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","KC_TRNS","MO(1)","KC_TRNS","KC_TRNS","KC_TRNS"] + ] +} +``` + +As you can see the payload describes all aspects of a keyboard necessary to create and generate a firmware. Each layer is a single list of QMK keycodes the same length as the keyboard's `LAYOUT` macro. If a keyboard supports mulitple `LAYOUT` macros you can specify which macro to use. + +## Submitting a Compile Job + +To compile your keymap into a firmware simply POST your JSON to the `/v1/compile` endpoint. In the following example we've placed the JSON payload into a file named `json_data`. + +``` +$ curl -H "Content-Type: application/json" -X POST -d "$(< json_data)" http://api.qmk.fm/v1/compile +{ + "enqueued": true, + "job_id": "ea1514b3-bdfc-4a7b-9b5c-08752684f7f6" +} +``` + +## Checking The Status + +After submitting your keymap you can check the status using a simple HTTP GET call: + +``` +$ curl http://api.qmk.fm/v1/compile/ea1514b3-bdfc-4a7b-9b5c-08752684f7f6 +{ + "created_at": "Sat, 19 Aug 2017 21:39:12 GMT", + "enqueued_at": "Sat, 19 Aug 2017 21:39:12 GMT", + "id": "f5f9b992-73b4-479b-8236-df1deb37c163", + "status": "running", + "result": null +} +``` + +This shows us that the job has made it through the queue and is currently running. There are 5 possible statuses: + +* **failed**: Something about the compiling service has broken. +* **finished**: The compilation is complete and you should check `result` to see the results. +* **queued**: The keymap is waiting for a compilation server to become available. +* **running**: The compilation is in progress and should be complete soon. +* **unknown**: A serious error has occurred and you should [file a bug](https://github.com/qmk/qmk_compiler/issues). + +## Examining Finished Results + +Once your compile job has finished you'll check the `result` key. The value of this key is a hash containing several key bits of information: + +* `firmware_binary_url`: A list of URLs for the the flashable firmware +* `firmware_keymap_url`: A list of URLs for the the `keymap.c` +* `firmware_source_url`: A list of URLs for the full firmware source code +* `output`: The stdout and stderr for this compile job. Errors will be found here. diff --git a/docs/api_overview.md b/docs/api_overview.md new file mode 100644 index 00000000000..91d317f06bc --- /dev/null +++ b/docs/api_overview.md @@ -0,0 +1,15 @@ +# QMK API + +The QMK API provides an asynchronous API that Web and GUI tools can use to compile arbitrary keymaps for any keyboard supported by [QMK](http://qmk.fm/). The stock keymap template supports all QMK keycodes that do not require supporting C code. Keyboard maintainers can supply their own custom templates to enable more functionality. + +## App Developers + +If you are an app developer interested in using this API in your application you should head over to [Using The API](api_docs.md). + +## Keyboard Maintainers + +If you would like to enhance your keyboard's support in the QMK Compiler API head over to the [Keyboard Support](reference_configurator_support.md) section. + +## Backend Developers + +If you are interested in working on the API itself you should start by setting up a [Development Environment](api_development_environment.md), then check out [Hacking On The API](api_development_overview.md).