Drone Documentation Release 0.1 - Brad Rydzewski
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Drone Documentation Release 0.1 Brad Rydzewski January 11, 2015
Contents 1 Installation 3 1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Running . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 From Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Enabling SSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.6 Proxy Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Setup 7 2.1 Configure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 GitHub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 BitBucket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Mail Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3 Deployment 9 3.1 Cloud Foundry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2 Engine Yard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.3 Git . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.4 Bash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.5 Heroku . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.6 Modulus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.7 Nodejitsu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.8 Open Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.9 Tsuru . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4 Publish 13 4.1 S3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2 Swift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.3 PyPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.4 NPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.5 Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.6 GitHub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5 Services 19 5.1 Available Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.2 Example: Using Postgres . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5.3 Tips for Using Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5.4 More Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 i
6 Contributing 23 6.1 Pull Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6.2 Mailing List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6.3 GitHub Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 7 Need Help? 25 ii
Drone Documentation, Release 0.1 Drone is a Continuous Integration platform built on Docker. Contents: Contents 1
Drone Documentation, Release 0.1 2 Contents
CHAPTER 1 Installation 1.1 Requirements Drone is tested on the following versions of Ubuntu: • Ubuntu Precise 12.04 (64-bit) • Ubuntu Raring 13.04 (64 bit) Drone requires the latest version of Docker (0.8) 1.2 Installation Drone is distributed as a debian package for easy installation: $ wget http://downloads.drone.io/master/drone.deb $ sudo dpkg -i drone.deb This will install the following files: • Drone server /usr/local/bin/droned • Drone client /usr/local/bin/drone • Drone startup script /etc/init/drone.conf • Drone sqlite database /var/lib/drone/drone.sqlite We recommend running Drone on a 2GB Digital Ocean Docker image. This is the fastest, easiest way to get up and running with Drone. This is also how we test Drone internally. 1.3 Running Drone is started automatically. You can start and stop Drone manually with the following commands: $ sudo start drone $ sudo stop drone 3
Drone Documentation, Release 0.1 1.4 From Source The project is hosted at https://github.com/drone/drone and can be installed manually. You will need the latest version of Go (1.2) and the following software packages: $ sudo apt-get install make mercurial git bzr libsqlite3-dev sqlite3 $ git clone git://github.com/drone/drone.git $ cd drone $ make deps $ make 1.5 Enabling SSL To enable SSL/TLS security on Drone you will need to edit the Upstart script configuration. This file is located at /etc/default/drone. You will need to change the DRONED_OPTS variable to have three things: • Use a different port (Usually --port=:443). • The full path to the SSL certificate (--sslcert=/path/to/my.crt) • The full path to the SSL certificate’s secret key (--sslkey=/path/to/my.key) Note: Your SSL key should be kept in a safe directory with restrictive permissions. On many Linux systems, /etc/ssl/private is used for this. Note: If your SSL certificate requires a bundle, it should be appended to the certificate file. E.g. if you have “my-cert.crt” and “my-bundle.crt” then cat my-cert.crt my-bundle.crt >> my-cert-bundle.crt’ with --sslcert=/path/to/my-cert-bundle.crt. If you do not know where to place the certificate, /etc/ssl/certs/ is a good candidate. Here is a complete example: # Upstart configuration file for droned. # Command line options: # # -datasource="drone.sqlite": # -driver="sqlite3": # -port=":8080": # -workers="4": # DRONED_OPTS="-port=:443 --sslkey=/path/to/my.key --sslcert=/path/to/my.crt" Once this file has been changed you will need to restart droned before these changes will take effect. 1.6 Proxy Server NOTE: using a proxy server is not really recommended. Drone serves most static content from a CDN and uses the Go standard library’s high-performance net/http package to serve dynamic content. If using Nginx to proxy traffic to Drone, please ensure you have version 1.3.13 or greater. You also need to configure nginx to proxy websocket connections: 4 Chapter 1. Installation
Drone Documentation, Release 0.1 # Proxy for websockets location = /feed { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header Host $http_host; proxy_pass http://127.0.0.1:8080; proxy_redirect off; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } You will also want to change the default port. You can pass the port as a command line argument --port=:8080 or you can change the default port in the /etc/default/drone file: # Upstart configuration file for droned. # Command line options: # # -datasource="drone.sqlite": # -driver="sqlite3": # -path="": # -port=":8080": # DRONED_OPTS="--port=:8080" 1.6. Proxy Server 5
Drone Documentation, Release 0.1 6 Chapter 1. Installation
CHAPTER 2 Setup This section assumes Drone is installed as a service and running on Port 80. 2.1 Configure First you need to enable one of the OAuth services in ‘config.toml‘ or using ENV variables. Then you will be able to login as admin and add other authorized users if you have disabled open registration. 2.2 GitHub Navigate to https://github.com/settings/applications and click the Register New Application button. Enter the follow- ing information into the form: Application Name Drone Homepage URL http://localhost Authorization Callback URL http://localhost/api/auth/github.com Click Register Application Copy and paste the Client ID and Client Secret into the Drone admin / settings screen. 2.3 BitBucket To allow Drone access to your BitBucket projects, you will need to authorize Drone as an “Integrated Application” within Bitbucket. This will generate OAuth keys that Drone uses to authenticate itself to Drone. Navigate to your BitBucket account settings: https://bitbucket.org/account/user/USER_NAME/. Click on Integrated applications and then press the Add consumer button. This will open up a dialog window asking for Name, Descrip- tion, and URL. Once you complete and submit this dialog, you will be given two pieces of information: • Key • Secret Now in Drone you can navigate to Drone’s settings and scroll to the section called Bitbucket OAuth Consumer Key and Secret and enter the key value in the first field, and the secret in the second field. 7
Drone Documentation, Release 0.1 Note: In addition to being able to grant BitBucket user access to drone, you can also associate Drone to a BitBucket Team. To generate keys for a team, navigate to the team’s page, click Manage team, and then click on the Integrated Applications item in the left-hand navigation. The first time you add a BitBucket repository to Drone, you will be asked to authorize the Drone to BitBucket con- nection. 2.4 Mail Server Setting up an email server is highly recommended. Drone uses email messages for the following: • account activation emails • password reset emails • team invitation emails • build result emails Navigate to http://localhost/account/admin/settings and enter your SMTP server details: 8 Chapter 2. Setup
CHAPTER 3 Deployment Drone can automate your deployment steps at the end of each successful build. Drone includes a number of deployment plugins that can be configured in the deploy section of the .drone.yml file. Deployments are skipped for: • Failed builds • Pull requests 3.1 Cloud Foundry Deploy to Cloud Foundry hosted service over cf tool verion 6. it assumes your provide the application manifest file manifest.yml under project root directory if no application name provided. deploy: cloudfoundry: target: https://api.run.pivotal.io username: my-cloudfoundry-username password: my-password organization: my-cloudfoundry-org space: development app: my-cloudfoundry-app target URL of the Cloud Controller in Cloud Foundry username your username password your password organization organization name where you want to deploy your application (optional) space space name in the organization where you want to deploy your application (optional) app name of the application to push (optional) 3.2 Engine Yard We are looking for volunteers to add this plugin. 9
Drone Documentation, Release 0.1 3.3 Git Deploy code via git push to a remote server. deploy: git: target: git@foo.com/bar.git branch: master force: false target name of the git remote server to push to branch destination branch, optional, defaults to master force true | false to use the git –force flag 3.4 Bash Deploy code via any bash command, for example with [Capistrano](https://github.com/capistrano/capistrano) or [Fab- ric](https://github.com/fabric/fabric) deploy: bash: command: bundle exec cap production deploy or deploy: bash: script: - ./bin/prepare_for_deploy.sh - ./bin/make_deploy.sh - ./bin/finish_deploy.sh command bash command that runs deploy script array of bash commands that run deploy 3.5 Heroku Deploy to the Heroku hosting service. deploy: heroku: app: my-heroku-app force: false app name of your heroku application force true | false to use the git –force flag 3.6 Modulus Deploy to the modulus.io hosting service. 10 Chapter 3. Deployment
Drone Documentation, Release 0.1 deploy: modulus: project: my-modulus-app token: 5f05189c project name of your modulus project token your modulus api token 3.7 Nodejitsu Deploy to the nodejitsu hosting service. deploy: nodejitsu: user: my-nodejitsure-username token: 5f05189c user your nodejitsu username token your nodejitsu api token 3.8 Open Shift We are looking for volunteers to add this plugin. 3.9 Tsuru Deploy to the Tsuru hosting service. deploy: tsuru: force: false remote: git@git.tsuru.io:my-tsuruapp.git force true | false to use the git –force flag (default: false). remote git remote of your tsuru application 3.7. Nodejitsu 11
Drone Documentation, Release 0.1 12 Chapter 3. Deployment
CHAPTER 4 Publish Drone can automate publishing files at the end of each successful build. Drone includes a number of publish plugins that can be configured in the publish section of the .drone.yml file. Publishes are skipped for: • Failed builds • Pull requests 4.1 S3 Publish files to Amazon S3 publish: s3: acl: public-read region: us-east-1 bucket: downloads.drone.io access_key: C24526974F365C3B secret_key: 2263c9751ed084a68df28fd2f658b127 source: /tmp/drone.deb target: latest/ acl Access control to apply to file region Region where the bucket resides that you are publishing to bucket Bucket to publish the file to access_key AWS Access key secret_key AWS Secret Key source Source file to publish target Destiantion publish location 4.2 Swift Publish files to OpenStack Swift or Rackspace CloudFiles 13
Drone Documentation, Release 0.1 publish: swift: username: someuser password: 030e39a1278a18828389b194b93211aa auth_url: https://identity.api.rackspacecloud.com/v2.0 region: DFW container: drone source: /tmp/drone.deb target: latest/drone.deb username OpenStack/Rackspace Username to authenticate with password OpenStack Keystone password or Rackspace API Key auth_url URL of the Keystone identitiy endpoint region Region where the container resides that you are publishing to container Container to publish the file to source Source file to publish target Destination publish location (only required when source is a single file) 4.3 PyPI Publish a Python package to pypi.python.org publish: pypi: username: someuser password: somepassword username PyPI username to authenticate with password PyPI password to authenticate with 4.4 NPM Publish a Node.js package to npm registry publish: npm: username: someuser email: someuser@example.com password: somepassword registry: https://somereg.example.com/ folder: my-node-project/ tag: latest username npm registry username to authenticate with email npm registry email to authenticate with password npm registry password to authenticate with registry npm registry URL. default to http://registry.npmjs.org/ (optional) folder a folder containing a package.json file (optional) 14 Chapter 4. Publish
Drone Documentation, Release 0.1 tag registers the published package with the given tag (optional) 4.5 Docker Publish a Docker image to a specified repo or registry. Supports the following configurations: • Private Docker Registry (unauthenticated) • Private Docker Registry (authenticated) e.g. login with username + push to docker.example.com/image:tag • Push to Docker Hub user ID e.g. username/image:tag • Push to Docker Hub company or group e.g. login with username but push to company/image:tag publish: docker: dockerfile: MyDockerFile docker_host tcp://docker.example.com:2375 docker_version: 1.3.0 registry_host: docker.example.com registry_protocol: https registry_login: true registry_login_uri: /registry/v1/ username: myuser password: mypassword email: myuser@example.com image_name: my-webapp tags: [0.1, latest] dockerfile The Dockerfile you want to use to build your final image. Default: ./Dockerfile in the root of your code- base. docker_host Docker server that you want to connect to for building/pushing your image. It has the same format than -H flag of docker binary. Note: This does not need to match the final destination/end-point for your image. docker_version The version of Docker Engine that is running on the remote Docker server (not the registry). registry_login_url (optional) The full login URI used to post authentication details (e.g. https://docker.company.com/v1/ ) registry_login (optional) Does your custom registry endpoint require login? Defaults to false Note: This is not applicable when pushing to Docker Hub, it will always require authentication. image_name The name you would like to give your image (excluding the image tag) tag (optional) A custom tag for this image. tags (optional) The tag(s) you would like to set for this image build. Will be combined with the “tag” field if both are specified. If no tags are specified in either field, the image will be tagged with the short git commit ID git rev-parse –short HEAD username (optional for private repositories) The username used to authenticate to the private registry or to Docker Hub password (optional for private repositories) Your authentication password email (optional for private repositories) Your email address keep_build (optional) Set to true if you would like to leave the final image on the docker_server used to build it. Default is false, which cleans up the build after successfully pushing to the registry. 4.5. Docker 15
Drone Documentation, Release 0.1 4.5.1 Example Configs Private Registry, no authentication publish: docker: docker_host: tcp://docker.example.com:1000 docker_version: 1.0 registry_login: false image_name: docker.example.com/my-webapp keep_builds: false tag: 0.1 Result: Image pushed to docker-registry.example.com/my-webapp:0.1 without login. Private Registry, Authenticated publish: docker: docker_host: tcp://docker.example.com:1000 docker_version: 1.0 registry_login_url: https://docker-registry.example.com/v1/ registry_login: true username: myuser password: mypassword email: myuser@example.com image_name: docker-registry.example.com/my-webapp keep_builds: false Result: Image pushed to docker-registry.example.com/my-webapp:$(git rev-parse –short HEAD) using myuser ac- count. Docker Hub, Push to Personal Account .. code-block:: console publish: docker: docker_host: tcp://docker.example.com:1000 docker_version: 1.0 username: myuser pass- word: mypassword email: myuser@example.com image_name: my-webapp keep_builds: false Result: Image pushed to Docker Hub as myuser/my-webapp:$(git rev-parse –short HEAD) using myuser account. Docker Hub, Push to Shared Repository publish: docker: docker_host: tcp://docker.example.com:1000 docker_version: 1.0 username: myuser password: mypassword email: myuser@example.com image_name: mycompany/my-webapp keep_builds: false tags: [0.1, 0.2] Result: Image pushed to Docker Hub as mycompany/image:0.1 and mycompany/image:0.2 using myuser account. publish: docker: docker_host: tcp://docker.example.com:1000 docker_version: 1.0 username: myuser pass- word: mypassword email: myuser@example.com image_name: mycompany/my-webapp keep_builds: false tag: 0.1-latest tags: [0.1, dev-latest] 16 Chapter 4. Publish
Drone Documentation, Release 0.1 Result: Image pushed to Docker Hub as mycompany/image:0.1, mycompany/image:0.1-latest and mycompany/image:dev-latest using myuser account. 4.6 GitHub Create a GitHub release. publish: github: branch: master script: - make embed - make release artifacts: - release/ tag: v$(cat VERSION) token: {{github_token}} user: drone repo: drone script Optional list of commands to run to prepare a release. artifacts List of files or directories to release. tag Required name of the tag to create for this release. If the tag already exists, the plugin will do nothing. name The name of the release. Defaults to tag. description Description of the release. Defaults to empty. draft: true/false identifier allowed on GitHub releases. Defaults to false. prerelease: true/false identifier allowed on GitHub releases. Defaults to false. token: Required OAuth token to use when interacting with the GitHub API. The token must have “repo” access, and the user associated with the token must have read and write access to the repo. user: The user or organization for the repository you’d like to publish to. repo: The name of the respository to publish to. branch: Restrict this plugin to only run on commits to this branch. Defaults to “master”. 4.6. GitHub 17
Drone Documentation, Release 0.1 18 Chapter 4. Publish
CHAPTER 5 Services Drone provides a facility for connecting an application to one or more service containers. A service is any Docker container that provides facilities that your primary container needs to use. The most common examples are databases (like mysql, postgres, mongo), log services (log4j), and search servers (elasticsearch). These and many others are provided out of the box. But just as you can use any image as a basis for your Drone build, you can also use any image as a service. This section explains how to connect a service to your build using the .drone.yml file. 5.1 Available Services Like build images, Drone comes with many built-in service images. And these, too, have short alias names. The following services are built-in: • cassandra (relateiq/cassandra) • couchdb (bradrydzewski/couchdb:1.5) • elasticsearch:0.90 (bradrydzewski/elasticsearch:0.90) • neo4j (bradrydzewski/neo4j:1.9) • memcached (bradrydzewski/memcached) • mongodb (bradrydzewski/mongodb:2.4) • mysql (bradrydzewski/mysql:5.5) • postgres (bradrydzewski/postgres:9.1) • rabbitmq (bradrydzewski/rabbitmq:3.2) • redis (bradrydzewski/redis:2.8) • riak (guillermo/riak) • zookeeper (jplock/zookeeper:3.4.5) NOTE: Services are added regularly, and many services have more than one version available. If what you are looking for is not in the list above, you may want to check the GitHub project at https://github.com/drone/drone. 19
Drone Documentation, Release 0.1 5.2 Example: Using Postgres Here is an example that uses the Postgres database as a Drone service. image: go1.2 services: - postgres script: - createdb -h localhost -U postgres drone - psql -U postgres -h localhost -c "SELECT 1" drone The example above does the following: • Uses the go1.2 image. Goose is built in Go. • Declares one service: postgres, which will create a new Postgres container and map it to our build container. • Creates a new database called drone using the Postgres command createdb. • Runs a very simple SELECT statement. Importantly, from the examples above, we can see the primary information that we can use to connect application code to Postgres: • Default user: postgres • Default password: none • Host: localhost • Port: 5432 • SSL: disabled (Some clients default to enabling SSL). Why point to localhost when Postgres is running in a different container? This is a feature of many of Drone’s built-in images. They are configured to proxy connections from the local host to the correct services containers. Once a project like this is pushed, the output should look something like this” $ git clone --depth=50 --recursive --branch=master https://bitbucket.org/technosophos/drone-postgres- Cloning into ’/var/cache/drone/src/bitbucket.org/technosophos/drone-postgres-example’... $ git checkout -qf c7e9917648ebcd3f5f5e622bc3502f41caf7cd64 $ createdb -h localhost -U postgres drone $ psql -U postgres -h localhost -c "SELECT 1" drone ?column? ---------- 1 (1 row) $ exit 0 Note that in the last few lines we can see the output of the query from Postgres. Using the same connection information presented above, you can configure your application to connect to the service instance and use it during the Drone run. 5.3 Tips for Using Services For the most part, the services that Drone provides are as close to their default configuration as possible. For example, above we saw that the postgres service uses the default user name (postgres) and no password, and attaches to the default port. 20 Chapter 5. Services
Drone Documentation, Release 0.1 If you would like to inspect a service, you may start it up as a plain Docker container (See the Docker example here: http://docs.docker.io/examples/postgresql_service/). Remember that you can create your own services. Remember also that data is not persisted from one build to another. In our example above, each time the build runs, we will have to create the new database. 5.4 More Information The Drone.io database guide (http://docs.drone.io/databases.html) contains a list of services and their configurations. 5.4. More Information 21
Drone Documentation, Release 0.1 22 Chapter 5. Services
CHAPTER 6 Contributing 6.1 Pull Requests We are always thrilled to receive pull requests, and do our best to process them as fast as possible. Not sure if that typo is worth a pull request? Do it! We will appreciate it. If your pull request is not accepted on the first try, don’t be discouraged! If there’s a problem with the implementation, hopefully you received feedback on what to improve. We’re trying very hard to keep Drone lean and focused. We don’t want it to do everything for everybody. This means that we might decide against incorporating a new feature. 6.2 Mailing List We recommend discussing your plans on the mailing list before starting to code - especially for more ambitious contributions. This gives other contributors a chance to point you in the right direction, give feedback on your design, and maybe point out if someone else is working on the same thing. We may close your pull request if not first discussed on the mailing list. We aren’t doing this to be jerks. We are doing this to prevent people from spending large amounts of time on changes that may need to be designed or architected in a specific way, or may not align with the vision of the project. 6.3 GitHub Issues Any significant improvement should be documented as a GitHub issue before anybody starts working on it. ...but check for existing issues first! Please take a moment to check that an issue doesn’t already exist documenting your bug report or improvement proposal. If it does, it never hurts to add a quick “+1” or “I have this problem too”. This will help prioritize the most common problems and requests. 23
Drone Documentation, Release 0.1 24 Chapter 6. Contributing
CHAPTER 7 Need Help? There are several ways to get in touch: • drone-dev on google groups • #drone on irc.freenode.net • twitter • github • stackoverflow 25
You can also read