Getting Started Guide¶

Welcome to the getting started guide of fink. In this guide we will cover what fink is and how to use it to create beautiful infrastructure as code (IaC):

fink tools to manage AWS infrastructure using CloudFormation and CodeDeploy:

using AWS CloudFormation with fink.cloud
deploy your application on AWS EC2 with AWS CodeDeploy scripts and fink.code

fink contains two more tools to manage serverless infrastructure on AWS:

deploy and configure AWS Lambda with fink.lambda
deploy API Gateway and manage API keys with fink.api

All of these things you can do for different Environments (dev, stage, prod)

Infrastructure as code¶

Infrastructure as Code (IaC) is a type of IT infrastructure that teams can automatically provision through code, rather than using a manual GUI-centered process. Through defining your infrastructure as code you can apply similar tools and workflows like when working with your application code like:

version control
test
review
share
reuse (like creating test envs)
audit

Installation¶

Install Python¶

First of all you need to have Python installed. Python should be 2.7 or higher

On MacOS try to use preinstalled Python

Install pip and virtualenv¶

virtualenv is a tool to create isolated Python environments. virtualenv creates a folder which contains all the necessary executables to use the packages that a Python project would need.

MacOS¶

$ sudo pip install virtualenv --upgrade

Install fink and fink plugins¶

Install fink¶

First of all you need to create virtualenv and activate it. We recommend create virtualenv in the same directory as a project, and add it to .gitignore. It’s pretty easy.

$ cd <project-folder>
$ virtualenv venv
$ source venv/bin/activate
$ pip install pip --upgrade # we always should have latest pip version in our virtualenv

fink needs some fink-glugins so you should install these together. fink-glugins are powerful tool to add features to fink without having to directly modify the fink core. The easiest way is to put the dependencies into a requirements_fink.txt file:

fink
fink.config-reader
fink.lookups
fink.bundler
fink.slack-integration
fink.cloud
fink.code
fink.lambda
fink.api

You can find more information about plugins in docs then

$ pip install -U -r requirements_fink.txt

To check that everything is good and fink installed just do:

$ fink version
fink version 0.1.418
fink commands:
 * fink.cloud 1.0.0
 * fink.code 1.0.0
 * fink.lambda 1.0.0
 * fink.api 1.0.0
fink plugins:
 * fink.config-reader version 1.0.0
 * fink.bundler version 1.0.0
 * fink.slack-integration version 1.0.0
 * fink.lookups version 1.0.0

Setting the ENV environment variable¶

For almost all fink commands you need to set the ENV environment variable. ENV is required to recognize which config file to use (fink_<env>.json). Usually you have a different config file for each environment (dev, stage, prod). You need to set ENV before running any fink command.

The following command will use the fink_dev.json config file

$ export PYTHONIOENCODING=UTF-8
$ ENV=dev cloud list 
...

Alternatively you can set the ENV variable for the duration of your terminal session. Set it like this:

$ export ENV=dev
$ cloud list
...

preparing your AWS account for deployments using fink tools¶

When you store your infrastructure as code in a version control system you do not want to keep the secret credentials with them for obvious reasons. Also you want to have the flexibility to deploy the same stack to different accounts without changing the code. In order to achieve this we store the credentials in AWS SSM parameter store:

essential parameters in your account

Another preparation step you need to take is a basestack (look for fink.base-sample-stack). This stack contains some essential infrastructure elements you need so your infrastructure deployments will work like roles, policies, an S3 artifact bucket etc. The basestack exposes some values using Outputs which can be looked up during deployments. So you deploy the basestack once per account and all other stack use this infrastructure.

fink.cloud¶

fink.cloud is a tool which help you to manage and deploy your infrastructure. AWS Cloudformation uses helps you configure and manage your AWS infrastructure as code. In cloud we have our cloudformation templates generated by troposphere. With cloud you can easily create (and configure) infrastructure for different environments (and AWS accounts) like for example (dev, stage, prod).

Create your first stack with cloud¶

First of all you need to create two files:

cloudformation.py - here you will describe your infrastructure using troposphere
fink_(dev|stage|prod) - settings for your ENV in json format, needs to include all parameters for the cloudformation template + stack name. You should have separate config for each ENV.

Let’s create a simple fink_dev.json (please change all values according your AWS account):

{
  "cloud": {
    "stack": {
      "StackName": "fink.superc-sample-stack"
    },
    "parameters": {
      "VPCId": "lookup:stack:<stack-name>:DefaultVPCId",
      "ScaleMinCapacity": "1",
      "ScaleMaxCapacity": "1",
      "InstanceType": "t2.micro",
      "DefaultInstancePolicyARN": "lookup:stack:<stack-name>:DefaultInstancePolicyARN",
      "AMI": "lookup:parameter:base_ami"
    }
  }
}

The values for VPCId and DefaultInstancePolicyARN are filled by by the fink-lookups which then will be used in the template. The fink.lookups plugin will search the outputs in the CloudFormation stack (as mentioned in the config).

Instead of <stack-name> you should provide your stack name or use hardcoded value(not recommended). It’s time to create our first Infrastructure as Code. Let’s do this. Here is a simple cloudformation.py script. Use it as a template for creating your infrastructure.

Deploy a stack to AWS¶

Before running a deployment we need to set some necessary ENV variables. Remember: You need this ENV variables exported each time before running any fink command.

$ export ENV=dev
$ export AWS_DEFAULT_PROFILE=fink # Default profile.
$ export AWS_DEFAULT_REGION=eu-west-1
$ export PYTHONIOENCODING=UTF-8

Run your first infrastructure deployment. It’s really easy:

$ cloud deploy

Kumo deploy output More information about fink.cloud can be found in docs

fink.lambda¶

fink.lambda will help you to deploy, manage and control AWS Lambda functions. Runtimes supported by lambda are: nodejs4.3, nodejs6.10, python2.7, python3.6

Deploy a simple AWS Lambda function¶

Create a fink_(dev|stage|prod).json file (please change all values according your AWS account):

"lambda": {
  "bundling": {
    "folders": [
        {
            "source": "./node_modules",
            "target": "./node_modules"
        }
    ],
    "zip": "bundle.zip"
  },
  "lambda": {
    "name" = "jenkins-fink-lifecycle-for-lambda",
    "description" = "lambda test for lambda",
    "role" = "lookup:stack:<stack-name>:LambdaArnForDeploy",
    "handlerFunction" = "handler.handle",
    "handlerFile" = "handler.py",
    "timeout" = "300",
    "memorySize" = "256",
    "vpc": {
            "subnetIds": [
                "lookup:stack:<stack-name>:LambdaSubnetIda",
                "lookup:stack:<stack-name>:LambdaSubnetIdb",
                "lookup:stack:<stack-name>:LambdaSubnetIdc"
            ],
        }
  }
}

then do:

$ lambda deploy

More information about fink.lambda can be found in docs

fink.code¶

fink.code will help you to deploy your application using AWS CodeDeploy. code will create an artifact bundle file and upload it to s3 which contains all files that you have in your codedeploy folder. Create the fink_(dev|stage|prod).json file (please change all values according your AWS account):

"code": {
  "codedeploy": {
    "applicationName": "lookup:stack:fink-sample-stack:applicationName",
    "deploymentGroupName": "lookup:stack:fink-sample-stack:DeploymentGroup",
    "deploymentConfigName": "lookup:stack:fink-sample-stack:DeploymentConfig",
    "artifactsBucket": "lookup:stack:<stack-name>:s3DeploymentBucket"
  }
}

then do:

$ code deploy

More information about fink.code can be found in docs

fink.api¶

fink.api is a tool that will help you to deploy and manage your API with AWS API Gateway. All you need is to put your swagger.yml into the same folder as a fink_(dev|stage|prod).json file. Also, add some new configs into it (please change all values according your AWS account):

"api": {
    "api": {
        "apiKey": "xxxxxxxxxxxxxx",
        "description": "Gcdt sample API based on dp api-mock",
        "name": "jenkins-fink-sample-api-dev",
        "targetStage": "mock"
    }
}

More information about fink.api can be found in docs