Cloud Native OCI

ATP + OCI Service Broker


Created with ❤ by Oracle A-Team

Table of Contents

Architecture Overview

OCI Service Broker Architecture

Lab Overview

  • Installing Service Catalog
  • Deploying OCI Service Broker
  • Provisioning ATP
  • Binding to created ATP (wallet auto-creation)
  • Deploying catalogue app

Setup Service Broker

  • Install Service Catalog
  • Create Credentials
  • Add OCI Service Broker
  • Register Cluster Broker
  • Verify
  • 👍 Good news! You already installed svc-cat/catalog as part of the umbrella chart during setup. Verify that the chart was installed:
    kubectl get pod -A -l chart=catalog-0.3.0
    ⚠️ If Service Catalog is not installed, please perform the following steps
    Create namespace for supporting services:
    kubectl create namespace mushop-utilities
    Add the Kubernetes Service Catalog helm repository:
    helm repo add svc-cat https://svc-catalog-charts.storage.googleapis.com
    Install the Kubernetes Service Catalog helm chart:
    helm install catalog svc-cat/catalog \
      --namespace mushop-utilities 
    ℹ️ Please note that the above command will deploy the OCI Service Broker using an embedded etcd instance. It is not recommended to deploy the OCI Service Broker using an embedded etcd instance and tls disabled in production environments, instead a separate etcd cluster should be setup and used by the OSB.
    The open source etcd operator project or a commercial offering may be used to setup a production quality etcd cluster. The recommended setup can be found here
  • The OCI Service Broker for Kubernetes requires tenancy credentials to provision and manage services and resources.
    Create a secret containing these values:
    kubectl create secret generic oci-credentials \
      --namespace mushop-utilities \
      --from-literal=tenancy=<TENANCY_OCID> \
      --from-literal=user=<USER_OCID> \
      --from-literal=region=<USER_OCI_REGION> \
      --from-literal=fingerprint=<PUBLIC_API_KEY_FINGERPRINT> \
      --from-literal=passphrase=<PRIVATE_API_KEY_PASSPHRASE> \
      --from-file=privatekey=<PATH_OF_PRIVATE_API_KEY>
    ⚠️ NOTE: The account used must have sufficient policies in IAM to manage resources provisioned with Service Broker
  • Next, deploy the OCI service broker on your cluster. This is done with the Oracle OCI Service Broker helm chart.
    helm install oci-broker https://github.com/oracle/oci-service-broker/releases/download/v1.5.0/oci-service-broker-1.5.0.tgz \
      --namespace mushop-utilities \
      --set ociCredentials.secretName=oci-credentials \
      --set storage.etcd.useEmbedded=true \
      --set tls.enabled=false
    ⚠️ Note the secretName=oci-credentials secret created before.
  • Lastly, establish the link between Service Catalog and the OCI Service Broker implementation by creating a ClusterServiceBroker resource.
    Service Broker
    Service Catalog

    Create file oci-service-broker.yaml with the following:

    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ClusterServiceBroker
    metadata:
      name: oci-service-broker
    spec:
      url: http://oci-broker.mushop-utilities:8080
    kubectl create -f oci-service-broker.yaml
  • Verify that the Service Broker is available and ready:
    kubectl get clusterservicebrokers -o 'custom-columns=BROKER:.metadata.name,STATUS:.status.conditions[0].reason'
    BROKER                STATUS
    oci-service-broker    FetchedCatalog
    Review the status, service and plans available:
    kubectl get clusterservicebrokers
    kubectl get clusterserviceclasses -o=custom-columns=CLASS:.spec.externalName,DESCRIPTION:.spec.description,CLASS\ ID:.metadata.name
    kubectl get clusterserviceplans -o=custom-columns=PLAN:.spec.externalName,DESCRIPTION:.spec.description,CLASS\ ID:.spec.clusterServiceClassRef.name
        NAME     NAMESPACE          CLASS                       DESCRIPTION
    +----------+-----------+----------------------+-------------------------------------+
      standard               atp-service            OCI Autonomous Transaction Processing
      archive                object-store-service   An Archive type Object Storage
      standard               object-store-service   A Standard type Object Storage
      standard               adw-service            OCI Autonomous Data Warehouse
      standard               oss-service            Oracle Streaming Service

Hands-on Service Broker

Follow the steps outlined here to provision an ATP instance and resolve a binding secret with its DB Connection Wallet:

  • 🔐Notice
  • Provision ATP
  • Service Binding
  • Run MuShop
  • Under the Hood
  • Full Automation
  • ⚠️ NOTE: The sample files in this exercise are using plain password instead of secrets for the sake of simplicity.

    For detailed instructions using proper secrets, please refer to the mushop source repo, under src/catalogue/kubernetes/ folder.

  • Resources provisioned (or claimed) by Service Broker are called Service Instances. Create an ATP Service Instance as follows:

    Create the file catalogue-oadb-instance.yaml with the following contents:

    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ServiceInstance
    metadata:
      name: catalogue-db-dev
    spec:
      clusterServiceClassExternalName: atp-service
      clusterServicePlanExternalName: standard
      parameters:
        name: "MuShop Catalogue DB - dev"
        compartmentId: #[target compartment id. e.g.: ocid1.compartment.oc1..aaaaaaa...]
        dbName: cataloguedev
        cpuCount: 1
        storageSizeTBs: 1
        password: s123456789S@
        licenseType: BYOL
        autoScaling: false
    kubectl create -f catalogue-oadb-instance.yaml
    kubectl get serviceinstances
  • When Service Instances finish Provisioning, secrets in the form of a ServiceBinding will be created, and become available for application use.

    Create a file catalogue-oadb-binding.yaml with the following contents:

    apiVersion: servicecatalog.k8s.io/v1beta1
    kind: ServiceBinding
    metadata:
      name: catalogue-oadb-wallet-binding
    spec:
      instanceRef:
        name: catalogue-db-dev
      parameters:
        walletPassword: "Welcome_123"
    kubectl create -f catalogue-oadb-binding.yaml
    kubectl get servicebindings
    Once the binding shows READY, a secret will also be available. For ATP, this secret is in the form of a Base64 encoded DB Connection Wallet:
    kubectl get secret catalogue-oadb-wallet-binding -o yaml
  • A new ATP instance is now ready and available. The next step is to configure access secrets, and (re)deploy the application.
    ⚠️ NOTE: it is assumed that a MuShop chart configuration: myvalues.yaml already exists.
    1. Remove a previous deployment (if applicable):
      helm delete mushop
    2. Create catalogue-oadb-admin secret:
      kubectl create secret generic catalogue-oadb-admin \
        --from-literal=oadb_admin_pw='s123456789S@'
    3. Define catalogue-oadb-connection secret:
      kubectl create secret generic catalogue-oadb-connection \
        --from-literal=oadb_service=cataloguedev_tp \
        --from-literal=oadb_user=CATALOGUE_USER  \
        --from-literal=oadb_pw='default_Password1'
    4. From the source code, traverse into the helm chart deployment directory:
      cd deploy/complete/helm-chart
    5. Deploy MuShop with a flag indicating that the catalogue service is using an OCI Service Broker binding: --set catalogue.osb.atp=true
      helm install mushop mushop \
        --set catalogue.osb.atp=true \
        -f myvalues.yaml
  • Once again, the use of helm adds some mystery to the ultimate deployment of the catalogue service. Inspect the following for additional information.
    Inspect the catalogue deployment:
    kubectl get deploy mushop-catalogue -o yaml
    And the db init pod (name will vary):
    kubectl logs mushop-catalogue-1.2-init-d475m
    Example deployment of catalogue service:
    # Source: mushop/charts/catalogue/templates/catalogue-deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: mushop-catalogue
    spec:
      replicas: 1
      #...
      template:
        #...
        spec:
          terminationGracePeriodSeconds: 5
          initContainers:
            # OSB Wallet Binding decoder
            - name: decode-binding
              image: oraclelinux:7-slim
              command: ["/bin/sh","-c"]
              args: 
              - for i in `ls -1 /tmp/wallet | grep -v user_name`; do cat /tmp/wallet/$i | base64 --decode > /wallet/$i; done; ls -l /wallet/*;
              volumeMounts:
                - name: wallet-binding
                  mountPath: /tmp/wallet
                  readOnly: true
                - name: wallet
                  mountPath: /wallet
                  readOnly: false
          containers:
            - name: catalogue
              image: "iad.ocir.io/oracle/ateam/mushop-catalogue:1.2"
              imagePullPolicy: Always
              #...
              env:
                - name: ZIPKIN
                  value: 
                - name: OADB_USER
                  valueFrom:
                    secretKeyRef:
                      name: catalogue-oadb-connection
                      key: oadb_user
                - name: OADB_PW
                  valueFrom:
                    secretKeyRef:
                      name: catalogue-oadb-connection
                      key: oadb_pw
                - name: OADB_SERVICE
                  valueFrom:
                    secretKeyRef:
                      name: catalogue-oadb-connection
                      key: oadb_service
              volumeMounts:
                - name: wallet
                  mountPath: /usr/lib/oracle/19.3/client64/lib/network/admin/
                  readOnly: true
              #...
          volumes:
            # Service Broker Wallet binding
            - name: wallet-binding
              secret:
                secretName: catalogue-oadb-wallet-binding
            - name: wallet
              emptyDir: {}
            # service init configMap
            - name: initdb
              configMap:
                name: mushop-catalogue-init
                items:
                - key: atp.init.sql
                  path: service.sql
  • 💯 The processes contained in this exercise may be automated through the use of another helm chart included in the repo:
    deploy/complete/helm-chart/provision
    The provision chart includes:
    • OCI Service Broker API Secret
    • OCI ClusterServiceBroker
    • Global ATP Instance/Binding
    • Optional service-specific instance(s)
    • Auto-generated *-oadb-admin secrets
    • Auto-generated *-oadb-connection secrets
    1. Remove previous deployment (if applicable):
      helm delete mushop
    2. Provision instance(s) and binding(s):
      helm install provision provision \
        --set skip.clusterBroker=true \
        --set global.osb.oss=false \
        --set global.osb.objectstorage=false \
        --set global.osb.compartmentId=<COMPARTMENT_OCID>
    3. ⏲️ Wait for servicebindings to be READY:
      kubectl get servicebindings -A
    4. Deploy MuShop again, now indicating the use of Service Broker ATP backing:
      helm install mushop mushop \
        --set global.osb.atp=true \
        --set tags.streaming=false

Resources

Version: 1.8.0
Build: 2022-02-17T05:02:17Z
© 2022, Oracle and/or its affiliates. All rights reserved.