Amazon Web Services Ocient Installation

this guide explains how to install an {{ocient}} system in {{aws}} for details about aws concepts, see these pages https //docs aws amazon com/awsec2/latest/userguide/concepts html https //docs aws amazon com/iam/latest/userguide/introduction html https //aws amazon com/blogs/apn/amazon vpc for on premises network engineers part one/ ocient supports deployment in aws for pilot or testing purposes, but this setup does not guarantee data durability stopping amazon {{ec2}} instances can result in permanent data loss the steps for deploying an ocient system in aws are prepare aws resources set up an initial instance create amazon machine images (ami) from the initial instance launch other instances follow the standard ocient installation procedure example configuration the table below shows the recommended instance types for each node type true 820,820left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type left unhandled content type this diagram shows an example of an ocient cluster in aws the ec2 nodes (sql, loader, and foundation) are deployed within a single subnet of an amazon {{vpc}} aws assigns each type of node to a separate security group ( sg1 , sg2 , sg3 ) virtual private cloud for loading data from an s3 bucket using a jdbc client prepare aws resources create and configure these aws resources the vpc and subnets for the ocient system security groups to access the endpoints for each node type for details about the network security configuration, see the docid\ p7pdeoov7rn876uxmb29f identity and access management (iam) roles if you are loading data from s3, the loader nodes require iam access to an s3 bucket node setup (sql role) for initial instance launch instance use this configuration for your ami configuration steps differ depending on whether your setup uses a single volume or multi volume ami operating system (os) to set up the ami, you can use any ocient supported os (see docid\ yn3ugtv0bcrttcgalkltr ) single volume ami if you use a single volume ami, specify this configuration increase the root volume to 128gb or more multi volume ami if you use a multi volume ami (e g , cis hardened rhel 9), use this configuration increase root volume to 30gb or more increase elastic block store (ebs) volume to 100gb or more this ebs volume supports key system directories in the image ( /home , /var , /var/log , /var/log/audit , /var/tmp ) instance type use r5dn metal or a similar instance type security group use one or more security groups with these rules allow ssh to the nodes allow communication internally between nodes allow access to sql node endpoints described in the docid\ p7pdeoov7rn876uxmb29f access shell connect to your instance using secure shell (ssh) for details, see https //docs aws amazon com/awsec2/latest/userguide/connect linux inst ssh html extend logical volume manager (lvm) volumes (multi volume ami only) this step applies only to multi volume amis if you are using a single volume instance, skip this step if you use a multi volume manager, extend the /home and /var lvm volumes and their filesystems to fill up the expanded ebs volumes these actions expand the lvm volume and the contained file system to accommodate the ocient package, logging, and metadata these code examples show how to extend lvm volumes for a cis {{rhel}} 9 image other ami types might require different sizing contact ocient support for the best sizing for your system for multi volume instances example resize the physical volumes of two drives to use their full capacity after expanding them (see step 1) sudo pvresize /dev/nvme0n1 sudo pvresize /dev/nvme1n1 extend local volumes add 66 percent of the available free space in vg 01 to the var vol logical volume add all (100 percent) of the remaining free space in vg 01 to the home vol logical volume sudo lvextend l +66%free /dev/vg 01/var vol sudo lvextend l +100%free /dev/vg 01/home vol extend the file system to use all available space on its underlying logical volume sudo xfs growfs /home sudo xfs growfs /var update packages update all your software packages to their latest versions and then reboot your instance for rhel compatible systems, use this command sudo dnf update for {{debian}} compatible systems, use this command sudo apt update reboot after the update sudo reboot install the ocient package copy over the ocient rpm or deb package and install it for rhel compatible systems, use this command sudo dnf install /ocient release xx x x xxxxxxxxxxxxxx x86 64 rpm for debian compatible systems, use this command sudo apt install /ocient release xx x x xxxxxxxx xxxxxx xxxxxxxxxxxx amd64 deb set up kernel parameters use the ockernelparams utility to set up kernel parameters automatically, including the huge pages configuration repeat this step on other nodes sudo /opt/ocient/scripts/ockernelparams node role sql reboot the system for the parameters to take effect sudo reboot check that the local storage drive is attached to the uio or vfio driver after reboot (the ocient package installs a service that runs on startup to do this) sudo /opt/ocient/scripts/nvme driver util sh for examples of attaching drivers to the nvme drives, see docid\ ja2vhl pd5rbbjug6wk 8 configure the firewall if you are not using an os level firewall, skip this step if your base ami includes a system firewall, you must configure rules that explicitly allow required network communication for your ocient deployment for details, see docid\ p7pdeoov7rn876uxmb29f required os firewall rules allow all necessary ports and protocols between ocient nodes by opening all tcp/udp ports within the private network range (e g , 10 0 0 0/16 ) or, allowing known ocient ports allow external access where needed by enabling ssh access (port 22) from your administrator ip range allowing client access to sql endpoints (for example, port 13101 or as specified in your setup) opening any additional ports required for monitoring or management tools for a list of required ports, see docid\ p7pdeoov7rn876uxmb29f create ami after your initial node is fully configured, you must replicate the setup process for the remaining nodes in your cluster this action ensures consistency and allows for proper internal communication between nodes for details about creating an ami, see https //docs aws amazon com/toolkit for visual studio/latest/user guide/tkv create ami from instance html set up remaining nodes to launch the remaining instances go through this process for each of your remaining nodes set up parameters launch the remaining instances with these parameters ami — use the ami created in the docid\ manlskiheob5ecigwxs 1 step instance type — use i3en metal or an equivalent instance that offers local nvme ssds for high performance local storage has high throughput and network bandwidth for internal cluster communication security groups — ensure these security rules are in place in the aws security groups associated with the nodes allow ssh to the nodes allow internal communication between all ocient nodes allow access to endpoints described in the docid\ p7pdeoov7rn876uxmb29f access shell connect to your instance using secure shell (ssh) for details, see https //docs aws amazon com/awsec2/latest/userguide/connect linux inst ssh html set up kernel parameters use the ockernelparams utility to set up kernel parameters automatically, including the hugepages parameters this example specifies a foundation node foundation use a different node type as necessary sudo /opt/ocient/scripts/ockernelparams node role foundation bootstrap the ocient system complete the bootstrapping process for your ocient system for details, see docid\ oeajn22ev3lj9tp zmzma related links docid\ zncvnrhsf6fg1yvqk6mxt docid\ s0dywbqubbanzcnc4z9fx