Dashboard Framework Part 2: Running Shiny in AWS Fargate with CDK
This article is originally published at https://www.quantargo.com/blog
Dashboard Framework Part 2: Running Shiny in AWS Fargate with CDK
In the previous post we outlined the architecture of a dashboard framework to run dashboards based on multiple technologies including Shiny and Flask in production. We will now show how to run a basic Shiny dashboard in AWS Fargate behind an Application Load Balancer in less than 60 lines of CDK code. To define our stack in a reproducible manner we will make use of the Amazon Cloud Development Kit (CDK) with Typescript. Starting from a basic CDK stack we now specify the most important components of our stack:
- The Application Load Balancer (ALB) to route traffic to our dashboards.
- The Fargate cluster to run our dashboard tasks in a scalable manner.
The deployed stack will finally run an example Shiny dashboard behind an Application Load Balancer. Note that the resulting stack will only run one dashboard without encryption. We’ll implement these features as part of the next post. The resulting CDK code can also be downloaded from Github at https://github.com/quantargo/dashboards.
Prerequisites
To run the following code examples make sure to have
- an AWS Account
- a locally configured AWS account by running e.g.
aws configurewith the aws CLI - a local Node.js installation (version >= 14.15.0)
- Typescript:
npm -g install typescript -
CDK (version >= 2.0):
npm install -g aws-cdk
Initialize CDK and Deploy first App
To initialize a sample project we first create a project folder and within the folder execute cdk init:
mkdir dashboards cd dashboards cdk init app --language typescript
This command creates a new CDK Typescript project and installs all required packages. The following 2 files are relevant for stack development:
-
bin/dashboards.ts: Main file which initializes CDK stack class. You can explicitly set the environmentenvif you use a different account or region for deployment. -
lib/dashboards-stack.ts: CDK Stack class to which all components of our stack will be added.
Specify Application Load Balancer (ALB)
Next, we need to create an Application Load Balancer (ALB) within a new VPC which is responsible for secure connections and routing. We create a new VPC and add an internetFacing load balancer to it. This means that the load balancer will be accessible from the public internet and will therefore be placed into a public subnet. Within the lib/dashboards-stack.ts file we put the following lines:
// Put imports on top of the file
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'
// Put below lines within the DashboardsStack constructor
const vpc = new ec2.Vpc(this, 'MyVpc');
const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', {
vpc: vpc,
internetFacing: true,
loadBalancerName: 'DashboardBalancer'
});Specify Dashboard Cluster
Next, we need to add an ECS cluster to our VPC to run our dashboards efficiently:
// Put imports on top of the file
import * as ecs from 'aws-cdk-lib/aws-ecs'
// Put below lines within the DashboardsStack constructor
const cluster = new ecs.Cluster(this, 'DashboardCluster', {
vpc: vpc
});Add First Fargate Task Definition
We can now add our first Fargate dashboard to the cluster by specifying a task definition. We use the rocker/shiny Docker container as an example running on port 3838. This also requires respective port mappings in the container definition. Additionally, we use half a virtual CPU (512)—1024 would equal a full one—and a memory size of 1024 MiB. By specifying the Fargate service we are already finished with the specification to run our first container in the cluster:
const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDefinition', {
cpu: 512,
memoryLimitMiB: 1024,
});
const port = 3838
const container = taskDefinition.addContainer('Container', {
image: ecs.ContainerImage.fromRegistry('rocker/shiny'),
portMappings: [{ containerPort: port }],
})
const service = new ecs.FargateService(this, 'FargateService', {
cluster: cluster,
taskDefinition: taskDefinition,
desiredCount: 1,
serviceName: 'FargateService'
})Put Service Behind ALB
Next, we put the Fargate service into an ALB target group so that traffic can be routed through the ALB:
const tg1 = new elbv2.ApplicationTargetGroup(this, 'TargetGroup', {
vpc: vpc,
targets: [service],
protocol: elbv2.ApplicationProtocol.HTTP,
stickinessCookieDuration: cdk.Duration.days(1),
port: port,
healthCheck: {
path: '/',
port: `${port}`
}
})Note that we added 2 parameters to the ALB target group definition:
- stickinessCookieDuration: Since Shiny sessions are stateful we need to prevent the ALB to switch instances (in case there are more) during a session. The session duration set to one day should be sufficient.
- healthCheck: The health check needs to specify the port (as string) and set to the container port 3838, as well.
Finally, we add an HTTP listener which directly forwards all incoming traffic to our dashboard:
const listener = lb.addListener(`HTTPListener`, {
port: 80,
defaultAction: elbv2.ListenerAction.forward([tg1])
})Deploy
Before deployment you should also bootstrap your CDK environment:
cdk bootstrap
Now the stack should be ready for deployment. As an extra step, you can now check if the stack can be successfully synthesized using
cdk synth
Any errors popping up during cdk synth need to be fixed immediately. By continously using cdk synth we make sure that the feedback cycles during development are as short as possible. If cdk synth is successful we can now run
cdk deploy
Finally, you should see the successful output message including the DashboardsStack.LoadBalancerDNSName which you can directly access through the browser:
Outputs: DashboardsStack.LoadBalancerDNSName = DashboardBalancer-<9-DIGIT-NUMBER>..elb.amazonaws.com Stack ARN: arn:aws:cloudformation: : :stack/DashboardsStack/ ✨ Total time: 297.67s
Destroy
If you don’t use the stack any more and to reduce cloud costs just run:
cdk destroy
Conclusion
We could show how to run your first basic Shiny dashboard behind an Application Load Balancer in very few lines of CDK Typescript code. In the next post we will cover end-to-end encryption through SSL/TLS and host-based routing to add multiple dashboards to the ALB.
Make code, not war! ✌️
Get in Touch
Interested in creating your own dashboard framework or other data science cloud stacks? Just get in touch:
E-Mail: [email protected]
Appendix - Full Code
The full CDK code stack for this post is available on Github.
Below you find the full code specifiying the stack from lib/dashboards-stack.ts:
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2'
import * as ecs from 'aws-cdk-lib/aws-ecs'
import * as cdk from 'aws-cdk-lib'
export class DashboardsStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const vpc = new ec2.Vpc(this, 'MyVpc');
const lb = new elbv2.ApplicationLoadBalancer(this, 'LB', {
vpc: vpc,
internetFacing: true,
loadBalancerName: 'DashboardBalancer'
});
const cluster = new ecs.Cluster(this, 'DashboardCluster', {
vpc: vpc
});
const taskDefinition = new ecs.FargateTaskDefinition(this, 'TaskDefinition', {
cpu: 512,
memoryLimitMiB: 1024,
});
const port = 3838
const container = taskDefinition.addContainer('Container', {
image: ecs.ContainerImage.fromRegistry('rocker/shiny'),
portMappings: [{ containerPort: port }],
})
const service = new ecs.FargateService(this, 'FargateService', {
cluster: cluster,
taskDefinition: taskDefinition,
desiredCount: 1,
serviceName: 'FargateService'
})
const tg1 = new elbv2.ApplicationTargetGroup(this, 'TargetGroup', {
vpc: vpc,
targets: [service],
protocol: elbv2.ApplicationProtocol.HTTP,
stickinessCookieDuration: cdk.Duration.days(1),
port: port,
healthCheck: {
path: '/',
port: `${port}`
}
})
const listener = lb.addListener(`HTTPListener`, {
port: 80,
defaultAction: elbv2.ListenerAction.forward([tg1])
})
new cdk.CfnOutput(this, 'LoadBalancerDNSName', { value: lb.loadBalancerDnsName });
}
}Thanks for visiting r-craft.org
This article is originally published at https://www.quantargo.com/blog
Please visit source website for post related comments.
