1 # XDS - X(cross) Development System Server
3 `xds-server` is a web server that allows user to remotely cross build applications.
5 The first goal is to provide a multi-platform cross development tool with
6 near-zero installation.
7 The second goal is to keep application sources locally (on user's machine) to
8 make it compatible with existing IT policies (e.g. corporate backup or SCM),
9 and let user to continue to work as usual (use his favorite editor,
10 keep performance while editing/browsing sources).
12 This powerful and portable webserver (written in [Go](https://golang.org))
13 exposes a REST interface over HTTP and also provides a Web dashboard to configure projects and execute _(for now)_ only basics commands.
15 `xds-server` uses [Syncthing](https://syncthing.net/) tool to synchronize
16 projects files from user machine to build server machine or container.
18 > **NOTE**: For now, only Syncthing sharing method is supported to synchronize
19 projects files. But in a near future and for restricted configurations, `xds-server`
20 will also support "standard" folder sharing (eg. nfs mount points or docker
23 > **SEE ALSO**: [xds-exec](https://github.com/iotbzh/xds-exec),
24 wrappers on `exec` commands that allows you to send commands to `xds-server`
25 and for example build your application from command-line or from your favorite
26 IDE (such as Netbeans or Visual Studio Code) through `xds-server`.
30 `xds-server` has been designed to easily compile and debug
31 [AGL](https://www.automotivelinux.org/) applications. That's why `xds-server` has
32 been integrated into AGL SDK docker container.
34 >**NOTE** For more info about AGL SDK docker container, please refer to
35 [AGL SDK Quick Setup](http://docs.automotivelinux.org/docs/getting_started/en/dev/reference/setup-sdk-environment.html)
39 Load the pre-build AGL SDK docker image including `xds-server`:
42 seb@laptop ~$ wget -O - http://iot.bzh/download/public/2017/XDS/docker/docker_agl_worker-xds-latest.tar.xz | docker load
47 You should get `docker.automotivelinux.org/agl/worker-xds:X.Y` image
50 # List image that we just built
51 seb@laptop ~$ docker images | grep worker-xds
53 docker.automotivelinux.org/agl/worker-xds 3.99.1 786d65b2792c 6 days ago 602MB
56 ### Start xds-server within the container
58 Use provided script to create a new docker image and start a new container:
62 seb@laptop ~$ wget https://raw.githubusercontent.com/iotbzh/xds-server/master/scripts/xds-docker-create-container.sh
64 # Create new XDS worker container
65 seb@laptop ~$ bash ./xds-docker-create-container.sh
67 # Check that new container is running
68 seb@laptop ~$ docker ps | grep worker-xds
70 b985d81af40c docker.automotivelinux.org/agl/worker-xds:3.99.1 "/usr/bin/wait_for..." 6 days ago Up 4 hours 0.0.0.0:8000->8000/tcp, 0.0.0.0:69->69/udp, 0.0.0.0:10809->10809/tcp, 0.0.0.0:2222->22/tcp agl-xds-seb@laptop-0-seb
73 This container (ID=0) exposes following ports:
75 - 8000 : `xds-server` to serve XDS Dashboard
79 #### Manually setup docker user id
81 If you plan to **use path-mapping sharing type for your projects**, you need to have the same user id and group id inside and outside docker. By default user and group name inside docker is set `devel` (id `1664`), use following commands to replace id `1664` with your user/group id:
83 # Set docker container name to use (usually agl-xds-xxx where xxx is USERNAME@MACHINENAME-IDX-NAME)
84 seb@laptop ~$ export CONTAINER_NAME=agl-xds-seb@laptop-0-seb
86 # First stop xds-server
87 seb@laptop ~$ docker exec ${CONTAINER_NAME} bash -c "systemctl stop xds-server"
89 # Change user and group id inside docker to match your ids
90 seb@laptop ~$ docker exec ${CONTAINER_NAME} bash -c "usermod -u $(id -u) devel"
91 seb@laptop ~$ docker exec ${CONTAINER_NAME} bash -c "groupmod -g $(id -g) devel"
93 # Update some files ownership
94 seb@laptop ~$ docker exec ${CONTAINER_NAME} bash -c "chown -R devel:devel /home/devel /tmp/xds*"
97 seb@laptop ~$ docker exec ${CONTAINER_NAME} bash -c "systemctl start xds-server"
100 ## Check if xds-server is running (open XDS Dashboard)
102 **`xds-server` is automatically started** as a service on container startup.
104 If the container is running on your localhost, you can access the web interface (what we call the "Dashboard"):
107 seb@laptop ~$ xdg-open http://localhost:8000
110 If needed you can status / stop / start it manually using following commands:
113 # Log into docker container
114 seb@laptop ~$ ssh -p 2222 devel@localhost
117 devel@docker ~$ sudo systemctl status xds-server.service
120 devel@docker ~$ sudo systemctl stop xds-server.service
123 devel@docker ~$ sudo systemctl start xds-server.service
125 # Get XDS server logs
126 devel@docker ~$ sudo journalctl --unit=xds-server.service --output=cat
129 ### Manually Start XDS server
131 XDS server is started as a service by Systemd.
134 /lib/systemd/system/xds-server.service
137 This Systemd service starts a bash script `/opt/AGL/xds/server/xds-server-start.sh`
139 If you needed you can change default setting by defining specific environment
140 variables in `/etc/default/xds-server`.
141 For example to control log level, just set LOG_LEVEL env variable knowing that
142 supported *level* are: panic, fatal, error, warn, info, debug.
145 seb@laptop ~$ ssh -p 2222 devel@localhost
146 devel@docker ~$ echo 'LOG_LEVEL=debug' | sudo tee --append /etc/default/xds-server > /dev/null
147 devel@docker ~$ sudo systemctl restart xds-server.service
148 devel@docker ~$ tail -f /tmp/xds-server/logs/xds-server.log
151 ### Install SDK cross-toolchain
153 `xds-server` uses cross-toolchain install into directory pointed by `sdkRootDir`
154 setting (see configuration section below for more details).
155 For now, you need to install manually SDK cross toolchain. There are not embedded
156 into docker image by default because the size of these tarballs is too big.
158 Use provided `install-agl-sdks` script, for example to install SDK for ARM64 and
162 seb@laptop ~$ ssh -p 2222 devel@localhost
164 # Install ARM64 SDK (automatic download)
165 devel@docker ~$ sudo /opt/AGL/xds/server/xds-utils/install-agl-sdks.sh --arch aarch64
167 # Install Intel corei7-64 SDK (using an SDK tarball that has been built or downloaded manually)
168 devel@docker ~$ sudo /opt/AGL/xds/server/xds-utils/install-agl-sdks.sh --arch corei7-64 --file /tmp/poky-agl-glibc-x86_64-agl-demo-platform-crosssdk-corei7-64-toolchain-
175 `xds-server` serves a web-application at [http://localhost:8000](http://localhost:8000)
176 when XDS server is running on your host. Just replace `localhost` by the host
177 name or ip when XDS server is running on another host. So you can now connect
178 your browser to this url and use what we call the **XDS dashboard**.
181 xdg-open http://localhost:8000
184 Then follow instructions provided by this dashboard, knowing that the first time
185 you need to download and start `xds-agent` on your local machine.
187 To download this tool, just click on download icon in dashboard configuration
188 page or download one of `xds-agent` released tarball: [https://github.com/iotbzh/xds-agent/releases](https://github.com/iotbzh/xds-agent/releases).
190 See also `xds-agent` [README file](https://github.com/iotbzh/xds-agent) for more details.
192 ## Build xds-server from scratch
196 - Install and setup [Go](https://golang.org/doc/install) version 1.7 or
197 higher to compile this tool.
198 - Install [npm](https://www.npmjs.com/)
199 - Install [gulp](http://gulpjs.com/)
204 sudo apt-get install golang npm curl git zip
205 sudo npm install -g gulp-cli
211 sudo zypper install go npm git curl zip
212 sudo npm install -g gulp-cli
215 Don't forget to open new user session after installing the packages.
221 Create a GOPATH variable(must be a full path):
224 export GOPATH=$(realpath ~/workspace_go)
227 Clone this repo into your `$GOPATH/src/github.com/iotbzh` and use delivered Makefile:
230 mkdir -p $GOPATH/src/github.com/iotbzh
231 cd $GOPATH/src/github.com/iotbzh
232 git clone https://github.com/iotbzh/xds-server.git
237 And to install `xds-server` (by default in `/opt/AGL/xds/server`):
243 >**NOTE:** Used `DESTDIR` to specify another install directory
245 >make install DESTDIR=$HOME/opt/xds-server
248 #### XDS docker image
250 As an alternative to a pre-build image, you can rebuild the container from scratch.
251 `xds-server` has been integrated as a flavour of AGL SDK docker image.
252 So to rebuild docker image just execute following commands:
255 # Clone docker-worker-generator git repo
256 git clone https://git.automotivelinux.org/AGL/docker-worker-generator
257 # Start build that will create a docker image
258 cd docker-worker-generator
259 make build FLAVOUR=xds
264 `xds-server` configuration is driven by a JSON config file (`config.json`).
266 Here is the logic to determine which `config.json` file will be used:
268 1. from command line option: `--config myConfig.json`
269 1. `$HOME/.xds-server/config.json` file
270 1. `/etc/xds-server/config.json` file
271 1. `<xds-server executable dir>/config.json` file
273 Supported fields in configuration file are (all fields are optional and example
274 below corresponds to the default values):
276 - **httpPort** : HTTP port of client webapp / dashboard
277 - **webAppDir** : location of client dashboard (default: webapp/dist)
278 - **shareRootDir** : root directory where projects will be copied
279 - **logsDir** : directory to store logs (eg. syncthing output)
280 - **sdkRootDir** : root directory where cross SDKs are installed
281 - **syncthing.binDir** : syncthing binaries directory (default: executable directory)
282 - **syncthing.home"** : syncthing home directory (usually .../syncthing-config)
283 - **syncthing.gui-address** : syncthing gui url (default http://localhost:8384)
284 - **syncthing.gui-apikey** : syncthing api-key to use (default auto-generated)
289 "webAppDir": "webapp/dist",
290 "shareRootDir": "${HOME}/.xds-server/projects",
291 "logsDir": "/tmp/logs",
292 "sdkRootDir": "/xdt/sdk",
295 "home": "${HOME}/.xds-server/syncthing-config",
296 "gui-address": "http://localhost:8384",
297 "gui-apikey": "123456789",
302 >**NOTE:** environment variables are supported by using `${MY_VAR}` syntax.
306 ### XDS server architecture
308 The server part is written in *Go* and web app / dashboard (client part) in
313 +-- bin/ where xds-server binary file will be built
315 +-- agent-config.json.in example of config.json file
317 +-- glide.yaml Go package dependency file
319 +-- lib/ sources of server part (Go)
321 +-- main.go main entry point of of Web server (Go)
323 +-- Makefile makefile including
325 +-- README.md this readme
327 +-- scripts/ hold various scripts used for installation or startup
329 +-- tools/ temporary directory to hold development tools (like glide)
331 +-- vendor/ temporary directory to hold Go dependencies packages
333 +-- webapp/ source client dashboard (Angular2 app)
336 Visual Studio Code launcher settings can be found into `.vscode/launch.json`.
340 - replace makefile by build.go to make Windows build support easier
342 - add more documentation
343 - add authentication / login (oauth) + HTTPS
344 - enable syncthing user/password + HTTPS