Single-Node Cloud Resilient Deployment Using AWS
Single-Node Cloud Resilient Deployment Using AWS
A Helm Chart ( GitHub, ArtifactHub) is available for our resiliency and high-availability deployment options. Be sure to read the deployment instructions in the associated README file before using the chart.
NEW in 3.93
Helm chart 93.0.0 and later includes optional ephemeral storage for
/nexus-data/logand/nexus-data/tmp. Use theephemeralStorageblock only when logs are forwarded externally because data is deleted when pods restart, are evicted, or are rescheduled.Helm chart 93.0.0 and later includes optional HPA support for clustered Nexus Repository HA deployments. Use the
hpablock to scale pods by CPU or memory after metrics-server is installed.Helm chart 93.0.0 and later supports Azure Workload Identity with External Secrets Operator for syncing Azure Key Vault secrets into Kubernetes Secrets. Configure this with the
serviceAccountandexternalsecretsblocks.
Use Cases
This reference architecture is designed to protect against the following scenarios:
- An AWS Availability Zone (AZ) outage within a single AWS region
- A node/server (i.e., EC2) failure
- A Nexus Repository service failure
You would use this architecture if you fit the following profiles:
- You are looking for a resilient Nexus Repository deployment option in AWS to reduce downtime.
- You would like to achieve automatic failover and fault tolerance as part of your deployment goals.
- You already have an Elastic Kubernetes Service (EKS) cluster set up as part of your deployment pipeline for your other in-house applications and would like to leverage the same for your Nexus Repository deployment.
- You have migrated or set up Nexus Repository with an external PostgreSQL database and want to fully reap the benefits of an externalized database setup.
- You do not need High Availability (HA) active-active mode.
- This model is compatible with the Community Edition. The included Helm chart are only available for Pro deployments, however they may be manually adjusted for CE deployments.
Requirements
To set up an environment like the one illustrated above and described in this section, you will need the following:
- Nexus Repository 3.33.0 or later
- An AWS account with permissions for accessing the following AWS services:
- Elastic Kubernetes Service (EKS)
- Relational Database Service (RDS) for PostgreSQL
- Application Load Balancer (ALB)
- CloudWatch
- Simple Storage Service (S3)
- Secrets Manager
Limitations
In this reference architecture, a maximum of one Nexus Repository instance is running simultaneously. Having more than one Nexus Repository instance replica will not work.
Setting Up the Architecture
Note
Unless otherwise specified, all steps detailed below are still required if you are planning to use the HA/Resiliency Helm Chart ( GitHub, ArtifactHub).
Step 1 - AWS EKS Cluster
Nexus Repository runs on an AWS EKS cluster spread across two or more AZs within a single AWS region. You can control the number of nodes by setting the min, max, and desired nodes parameters. EKS ensures that the desired number of Nexus Repository instances runs within the cluster. If something causes an instance or the node to go down, AWS will spin up another one. If an AZ becomes unavailable, AWS spins up new node(s) in the secondary AZ.
Begin by setting up the EKS cluster in the AWS web console, ensuring your nodes are spread across two or more AZs in one region. The EKS cluster must be created in a virtual private cloud (VPC) with two or more public or private subnets in different AZs. AWS provides instructions for managed nodes (i.e., EC2) in their documentation.
Validate the VPCs, subnets, and their network connections in the AWS VPC console under VPC → Your VPCs →
. Consider AWS's best practices for VPCs and subnets: In the AWS IAM console, validate that your nodegroup has an IAM role that has the following policies assigned (See AWS IAM Roles documentation):
- AmazonEKSWorkerNodePolicy
- AmazonEC2ContainerRegistryReadOnly
- AmazonEKS_CNI_Policy
- AmazonS3FullAccess (for S3 access)
If you want to use CloudWatch to stream logs, add the below policies:
- CloudWatchFullAccess
- CloudWatchAgentServerPolicy (to use AmazonCloudWatchAgent on servers)
In the AWS IAM console under IAM → Roles →
, ensure that the EKS cluster's role has AmazonEKSClusterPolicypermissions listed in the _Permissions_tab.Ensure that the role above is attached to the EKS cluster by checking the Cluster IAM role ARN section under EKS → Clusters → _
_in the AWS IAM console. Ensure that the cluster security group allows traffic into the cluster under the Inbound rules tab located in the VPC → Security Groups section of the AWS console.
Each Sonatype Nexus Repository instance requires at least 4 CPUs and 8GB memory. Ensure the EC2 nodes for the nodegroup are configured appropriately.
Ensure that there are a sufficient number of IP addresses available in the two subnets to account for the cluster, nodes, and other Kubernetes resources.
Ensure that the EKS API server endpoint is set to your desired setting: Public, Private, or both. This decision should be based on your requirements for exposing the EKS cluster outside of the VPC. Follow AWS documentation to control access to EKS clusters.
Enable DNS resolution and DNS hostnames for the VPC in the DNS settings section.
Under Security → Network ACLs →
, ensure that the Network ACL's inbound and outbound rules allow traffic into and from the subnets. (See the AWS NACL documentation.) Ensure that IAM OIDC identity provider for your cluster exists by using the following command:
aws iam list-open-id-connect-providers | grep $oidc_id | cut -d "/" -f4
If it does not exist, use the command below to create an IAM OIDC identity provider:
eksctl utils associate-iam-oidc-provider --cluster <cluster-name> --approve
If eksctl is not installed, create the IAM OIDC provider using the AWS web console from IAM → Identity providers.
13. In the AWS EKS console under EKS → Clusters →
- In the AWS CLI, run the following command; it should return the cluster information:
aws eks describe-cluster --name <cluster-name> --region <region-code>
- Run the kubectl command below to ensure that nodes are created:
kubectl get nodes
- In the AWS console under EC2 → Auto Scaling → Auto Scaling groups, ensure that the auto scaling group exists for the EKS cluster and has the desired number of nodes and AZs; this will have been created automatically with the EKS cluster.
Step 2 - Elastic File System Dynamic Provisioning
- Follow the AWS documentation to create an EFS in the AWS console.
- The EFS CSI driver creates an access endpoint when provisioning storage.
- The VPC used by you EFS must be the same as the one accessing your EKS cluster.
- Set the security group to allow incoming traffic on
port 2049from your EKS cluster security group. This is the security group used by all nodes in your EKS cluster.
- Create an IAM policy using the AWS-provided example and saving it as
iam_policy.json.
See Github file: iam-policy-example.json
- Use the following command to create the policy after saving your file:
aws iam create-policy \
--policy-name EFSCSIControllerIAMPolicy \
--policy-document file://iam-policy.json
- Create an IAM service account using a command like the following:
eksctl create iamserviceaccount \
--cluster=<cluster-name> \
--region=<region-name> \
--namespace=kube-system \
--name=efs-csi-controller-sa \
--override-existing-serviceaccounts \
--attach-policy-arn=arn:aws:iam::<aws-account-number>:policy/EFSCSIControllerIAMPolicy \
--approve
- Install the EFS CSI driver via helm charts using the following command. Update the parameters to match your deployment.
helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/
helm upgrade -i aws-efs-csi-driver aws-efs-csi-driver/aws-efs-csi-driver \
--namespace kube-system \
--set image.repository=602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/aws-efs-csi-driver \
--set controller.serviceAccount.create=false \
--set controller.serviceAccount.name=efs-csi-controller-sa
- Review the required updates to the
values.yamlbefore installing the Nexus Repository helm chart as documented in the README file.
Step 3 - Service Account
This is not required when using the HA Helm chart as this step is automatic.
- Run the Service Account YAML to establish the service account
Step 4 - AWS Aurora PostgreSQL Cluster
An Aurora PostgreSQL cluster containing three databases (one writer node and two replicas) spread across three AZs in the region where you've deployed your EKS provides an external database for Nexus Repository configurations and component metadata. We recommend creating these nodes in the same AZs as the EKS cluster.
- AWS provides instructions on creating an Aurora database cluster in their documentation.
Step 5 - AWS Load Balancer Controller
The AWS Load Balancer Controller allows you to provision an AWS ALB via an Ingress type specified in your Kubernetes deployment YAML file. This load balancer, which is provisioned in one of the public subnets specified when you create the cluster, allows you to reach the Nexus Repository pod from outside the EKS cluster. This is necessary because the nodes on which EKS runs the Nexus Repository pod are in private subnets and are otherwise unreachable from outside the EKS cluster.
Follow the AWS documentation to deploy the AWS LBC to your EKS cluster.
Step 6 - Kubernetes Namespace
Tip If you plan to use the HA Helm Chart ( GitHub, ArtifactHub), you do not need to perform the step below; the Helm chart will create this for you.
Step 7 - Licensing
There are two options to use for license management:
- AWS Secrets Manager
- Configure License in Helm Chart
Licensing Option 1 - AWS Secrets Manager
AWS Secrets Manager stores the license and the database connection properties. During a failover, the Secrets Manager retrieves the license when the container starts.
Licensing Option 2 - Configure License in Helm Chart
This option is only for those using the HA/Resiliency Helm Chart but not AWS Secrets Manager.
Step 8 - AWS CloudWatch (Optional but Recommended)
Tip If you plan to use the HA/Resiliency Helm Chart ( GitHub, ArtifactHub), you do not need to perform this step; the Helm chart will handle it for you.
Starting Your Deployment
Step 1 - StatefulSet
Run the StatefulSet YAML to start your Nexus Repository Pods.
Step 2 - Ingress YAML
Run the Ingress YAML to expose the service externally; this is required to allow you to communicate with the pods.
Step 12 Validation (Optional)
After running the YAML/Helm charts, use the following command to validate that the Ingress for the ALB was created
kubectl get ingress/ingress-nxrm-nexus -n <namespace>
The output should display the ALB's DNS name.