Elasticsearch SSO Implementation

Craig Pickles Posted 23 August 2021

Modis has implemented Amazon Elasticsearch to aggregate logs and provide a centralised location for monitoring and troubleshooting.

The use of the managed service reduces operational overheads but also leverages the tight integration with other AWS services. 

The Scenario

A state government customer has a number of microservices and applications that are logging to individual Cloud Watch log streams and investigating incidents or tracing bugs across dependencies is challenging and time consuming.

Modis has implemented Amazon Elasticsearch to aggregate logs and provide a centralised location for monitoring and troubleshooting. The use of the managed service reduces operational overheads but also leverages the tight integration with other AWS services.

A key requirement is to securely authenticate to Elasticsearch and Kibana using Single Sign On, rather than using basic authentication and the administrative burden of managing a separate user database. This also permissions for accessing indices and tenants in Kibana that can then be mapped to existing roles within the organisation.

Modis designed and implemented this solution in early 2021.

The Solution

Modis implemented Amazon Elasticsearch managed service in conjunction with a number of other AWS services for log ingestion. To provide a Single Sign On capability, AWS SSO and Fine-grained Access Control with SAML Authentication was implemented in Amazon Elasticsearch. This solution uses existing Active Directory groups and maps those to fine grained permissions in the Elasticsearch domain.

The architecture is illustrated as follows:

Elasticsearch SSO Implementation Architecture Diagram Illustration

In AWS SSO, Applications for Kibana are configured per environment and the SAML metadata for these applications are exported and stored in Secrets Manger for use during deployment. The steps are somewhat manual due to the limited functionality available in the CLI for the AWS SSO service.

The Applications are configured using the Idp-initiated URLs for each Elasticsearch domain, which allows the SSO landing page to be used as the point of entry for users.

Attribute

Value

Application ACS URL

https://<domain_endpoint>/_plugin/kibana/_opendistro/_security/saml/acs/idpinitiated

Application SAML audience

https://<domain_endpoint>

 

A specific set of mappings are defined to enable username, email address and Active Directory groups to be included in the SAML assertion.

A specific set of mappings are defined to enable username, email address and Active Directory groups to be included in the SAML assertion.

The Elasticsearch domain has VPC access enabled and is therefore only accessible from within the VPC and over Direct Connect. As such there is less emphasis on the Access Policy, which is permissive, with the focus instead on the use of Fine-Grained Access control and SAML authentication. SAML authentication imports the metadata from AWS SSO and the backend roles are mapped from an attribute in the SAML assertion:

Optional SAML Settings Image

 

The roles in AWS SSO are mapped as “backend roles” for the required Elasticsearch roles:

Backend Role

Elasticsearch Role/s

Developer

read_only, kibana_user

Administrator

security_manager, kibana_user, admin_access

Support

read_only, kibana_user

Release

read_only, kibana_user

 

Configuration of authentication in the Elasticsearch domain is done using the REST API, as are Tenants, visualizations, dashboards and the roles and permission that enable users to access them. Once of the principles of the solution was to automate as much of deployment as possible and security is no exception.

It is worthwhile noting that a current limitation using AWS SSO is that the groups mapped through as part of the SAML assertion is a UUID, rather than a named group. While this does not impact functionality, the use of ids is not as clean as a named role. As the AWS SSO managed service matures, there is an expectation that this will change, and the solution will be refined.


The Outcome

The Single Sign-On landing page is used to access Kibana in all environments, in addition to AWS accounts. The landing pages provides a centralized location for which to access all SSO integrated workloads, which is likely to be extended in the future.

Single Sign-On Image

The application in AWS SSO redirects to the organisation login page and after successful authentication, the user is logged into Kibana.

The application in AWS SSO redirects to the organisation login page and after successful authentication, the user is logged into Kibana image

The roles for the logged in user present the assigned permissions, and associated backend roles.

The roles for the logged in user present the assigned permissions, and associated backend roles.

This solution provides the customer with a managed service that will be maintained over time.

About Modis

At Modis we connect people, technology and businesses to the opportunities they need to thrive in a rapidly advancing world. With 1,300+ technology professionals across six locations (Sydney, Melbourne, Brisbane, Adelaide, Canberra and Perth), we work with our clients to deliver solutions and talent to transform technology portfolios, streamline business functions, drive innovation or enhance organisational capability. https://aws.modis.com

Find out how Modis can provide you with innovative AWS cloud based solutions and servicesModis has been an AWS Advanced Tier Partner since 2014. Modis' AWS Cloud Consulting services encompasses fundamentals of cyber security, fault tolerant digital system architecture, modernisation, traditional virtual machine or through to modern Serverless approaches, commercial off-the-shelf software operation to bespoke software development, delivered with high throughput, repeatable DevOps approaches to operations. With over half a decade of running critical authoritative government data sets that affects the lives of millions of citizens and the economies of the state, Modis has one of the most mature, experienced and recognised consulting service providers in the world. More importantly, we like to work very closely with our customers, not providing something to purchase, but taking a deep understanding of their business, and providing the recommendations and implementations to ensure a modern, efficient, reliable and secure environment for digital business systems.Contact us
Modis Australia | Animated map showing global locations
We operate around the world. Would you like to find out more about your local office?Find out about Modis