jessewei

Laboratory to bluemix, a cloud solution exercise to etl scheduler

Bluemix Container Lab @ Windows

Preparation

There are many configuration to enable Bulemix devops process. Here I show is the easiest way for lab.

  1. Install Docker@Windows
    Install Docker Toolbox
  2. Sign up for an IBM Bluemix account
  3. Install IBM Containers plugin
    Table 1. Two ways to install plugin @ Windows
    Operating system Command
    Windows 7 64-bit
    Linux 64-bit@VirtualBox

     

  4. Sign up for IBM Containers in IBM Bluemix
    • From the Bluemix dashboard, click Start Containers
    • You will be prompted to Set registry namespace.
    • Enter in a registry name, cannot be changed afterwards. This registry name is used across your account when using the IBM Containers service.

Read more…

Advertisements

Tensorflow@Windows

 

1. Preparation

Before start tutorial please check out Tensorflow install@Windows for environment preparation.

2. Git pull tutorial

There are tutorials on gitHub.

3. Mount tutorial to docker

docker -v /c/Users/jesse/Desktop/py/workspace/Tensorflow/tutorial:/notebooks/tutorial

Notice: /c/Users is the Docker default shared folder on Windows

 

 

 

Jupyter setup

Jupyter  is is a web-based interactive computational environment for creating IPython notebooks. Easy to shared tutorial or exercise information and snippet. It supported over 30 engine, include R, Java script, Java and  other Engine list.

My environment is Python 3.5.1:: Anaconda 4.0.0 (32-bit).

It is more convenient to install a Python distribution such as Anaconda; see here. Anaconda not only installs IPython and its requirements, but also a selection of frequently-used Python packages.

 

Jupyter shortcut

After install Anaconda, you need to setup Jupyter short cut as

C:\Anaconda3\python.exe C:\Anaconda3\cwp.py C:\Anaconda3 "C:/Anaconda3/python.exe" "C:/Anaconda3/Scripts/jupyter-notebook-script.py"

 

Create a Python 2 Engine

Or using conda, create a Python 2 environment:

conda create -n ipykernel_py2 python=2 ipykernel
activate ipykernel_py2    # On Windows, remove the word 'source'
python -m ipykernel install --user

You can find the engine “Python 2” is  added to your menu now:

Home - Google Chrome_2016-06-05_07-13-22

If you’re running Jupyter on Python 2 and want to set up a Python 3 kernel, follow the same steps, replacing step 2,3.

 

Tensorflow@Bluemix

 

Preparation

There are many configuration to enable Bulemix devops process. Here I show is the easiest way for lab.

  1. Install Docker@Windows
    Install Docker Toolbox
  2. Sign up for an IBM Bluemix account
  3. Install IBM Containers plugin
    Table 1. Two ways to install plugin @ Windows
    Operating system Command
    Windows 7 64-bit
    Linux 64-bit@VirtualBox

     

     

Read more…

Install IBM Containers plugin@Windows

Ref: Install IBM Container plugin@Step3

Install CloudFoundry@Windows

  1. Click download Windows 64 bit
  2. Unzip and Run
  3. Verify by
$ cf plugins

Change Language Setting

If your Windows are running on Traditional Chinese version, you need to change to English to make installation success.

Windows -> Control Panel -> Locale and Language

language_2016-03-27_22-32-16.jpg

Install IBM plug-in

Instead of using the depreciated IBM® Containers Extension (ice), use the Cloud Foundry plug-in for IBM Containers after 2016-Jan.

Install the IBM Containers Cloud Foundry plug-in download for your operating system by running one of the following commands.


Table 1. Cloud Foundry plug in commands
Operating System Command
Windows 64-bit

When the installation completes, an OK message is displayed.Verify the plug-in installation

Verify the plug-in installation

C:> cf plugins

Listing Installed Plugins…
OK

Plugin Name Version Command Name Command Help
IBM-Containers 0.8.826 ic IBM Containers plug-in

Logging in through CLI

Run Docker Quickstart

Windows -> Docker Quickstart Terminal

Log in to Bluemix through the Cloud Foundry CLI

$ cf login -a api.ng.bluemix.net
  1. For Email, enter the IBM ID that you use to log in to Bluemix.
  2. For Password, enter the password for the IBM ID that you use to log in to Bluemix. By entering your user name and password, your Bluemix organization and space are retrieved.
  3. Enter the number that represents one of your Bluemix organizations.
  4. Enter the number that represents one of your Bluemix spaces

Log in to the IBM Container service

$ cf ic login

 


 

 

Find and Pull Luigi

Pull the Louigi image on DockerHub

Login DockerHub and search image

httpshub.docker.jpg

Click filter and click detail to make sure the prefer image.

 

Pull the image

$ docker pull akursar/luigid
Using default tag: latest
...

Verify by running locally

$ docker run -d -p 8082:8082 --name lcd akursar/luigid

Verify Docker ip

$ ifconfig docker
docker0 Link encap:Ethernet HWaddr 02:XX:XX:XX:XX:e0 
 inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0

Access through browser

In your browser, access http://172.17.0.1:8082.

Stop local containers

$ docker stop lcd
$ docker rm lcd
 
 

 

 

Marketplace Platform as a Service

Ref: PaaS@wiki

The Marketplace Platform as a  Service, mkPaaS, is a model of PaaS. It is the model what my personal needs for daily operation. The mkPaaS not only allowing joint party develop, run, and manage applications, but also  allow them selling or buying service in this marketplace. Trading parties may exchanging software or service like appstore on mkPaaS, not only product, the platform also provides integrated and configurable multi-layer service.

I believe Docker  will change ecosystem of cloud . The software market would be restructure, allow software and service supply chain integrate to demand side. The boundary of  frontend and backend will become vaguer, because multiple layer microservices model come true. I believe the Coase viewpoint in new institutional economics , I think the market segment will changed when transactional cost rapid decline soon.

The mkPaaS would become the only form of PaaS, major form of software market, and the  final form of service market. Because the property owner of software or service could represent himself when transaction cost approach zero, the market have no boundary to those individual household. There are many opportunity left to parties who provide integration service on the marketplace platform, that’s  the 3rd party managed PaaS.

mkPaaS

The mkPaaS, will own by the one who trying to engage customer and supplier in minimum transaction cost. The price signal would clear to drive supply and demand make their decision.

The lifecycle of this arena should similar as product. At first cycle, some quick win service provider will become frontier. They should be the easy replicable role model because it is the critical marketing strategy of mkPaaS vendor. Only few winner in first cycle would  survive in next cycle when they found their strength is two sides of the same picture.

PowerPoint Slide Show - [je-docker-v1.jpg

The second cycle will come before first decline stage happen because who work hard at first cycle and lesson learnt. They will success due to  introduced new architectural pattern. The mkPaaS vendor who partner with those winner would be dominator in next decades because the allies will build an intelligent software demand/supply chain.

Run the web app

Run images the same way you did locally but without any worry of keeping your laptop on all day!

  • Check images are in your Bluemix hosted registry

     $ cf ic images
     REPOSITORY                                        TAG 
     registry.ng.bluemix.net/web1/lets-chat           latest              
     registry.ng.bluemix.net/web1/mongo               latest              
     
    
  • Run Mongo container

  Just like locally, except this time use cf ic instead of docker to point to Bluemix.

$ cf ic run --name lc-mongo -p 27017 -m 512 registry.ng.bluemix.net/web1/mongo

  • Show the running container instances

    Wait for a state of RUNNING before you proceed:

    $ cf ic ps
    CONTAINER ID        IMAGE                                                       
    7ebf51a3-35a        registry.ng.bluemix.net/web1/mongo:latest   ""
    
  • Run  Let’s Chat container

    $ cf ic run --name lets-chat --link lc-mongo:mongo -p 8080 -m 256 registry.ng.bluemix.net/web1/lets-chat

    a5dc5e0d-8eae-44a2-9f8d-548112bec250

    Show the running container instances. Wait for a state of RUNNING before you proceed:

    $ cf ic ps
    CONTAINER ID        IMAGE                                                                
    d368a598-69d        registry.eu-gb.bluemix.net/web1/lets-chat:latest
    7ebf51a3-35a        registry.eu-gb.bluemix.net/web1/mongo:latest       
  • Expose Let’s Chat container to the public Internet

    The IBM Containers command line tool will attempt to expose your container for you if you have room left in your Public IP Address quota.

    Check which IPs are available and then bind one to your running container.

    $ cf ic ip list
    Number of allocated public IP addresses:  2
    
    IpAddress        ContainerId   
    134.XXX.YYY.ZZ0       
    134.XXX.YYY.ZZ1
    

    If you have no available IP addresses in the response, you can request one:

    $ cf ic ip request
    Successfully requested ip 134.XXX.YYY.ZZZ
    

     

  • Bind an available IP address

    $ cf ic ip bind 134.XXX.YYY.ZZZ lets-chat
    OK
    The IP address was bound successfully
    

    Show running containers with bound IP information now visible:

    $ cf ic ps
    CONTAINER ID        IMAGE 
    d368a598-69d        registry.ng.bluemix.net/web1/lets-chat:latest   ""
    7ebf51a3-35a        registry.ng.bluemix.net/web1/mongo:latest       ""     
                 
  • Running app in browser

    At the IP you just bound. Remember to use port 8080!

Conclusion

Congratulations, you have successfully completed this IBM Containers lab!. In this lab, you learned how to tag and push local images to Bluemix, inspect pushed images for security vulnerabilities, and run hosted multi-container applications on IBM Containers.

Now you can take the hands-off approach to all your future application deployments!

Cleanup

If you plan to do another lab, you need to clean up your container instances. This can be done through the UI and the DELETE button on each container, or you can do this through the CLI with the cf ic rm -f [CONTAINER_NAME] command.

$ cf ic rm -f lets-chat
$ cf ic rm -f mongo
 

Verify security vulnerabilities

One of the fundamental aspects of Docker containers is reuse and the ability to base your containers on top of other containers. IBM Containers provides Vulnerability Advisor, a pre-integrated security scanning tool that will alert you of vulnerable images and can even be configured to prevent deployment of those images.

  1. Go to the Bluemix Dashboard and click on CATALOG.
  2. Hover over the purple icon for Mongo. This is the Mongo image that you pulled from the public DockerHub registry and pushed into your private registry.You will see a pop-up with the vulnerability assessment shown inline. This is a red/yellow/green scale. Your Mongo image should be a green status of Safe to Deploy.
  3. Click on the Mongo image and you are taken to the container deployment page. You won’t deploy your container from here, but you can see the vulnerability assessment in full detail.On the right side of the screen, you can see your image’s Vulnerability Assessment as well as your quota information. The icon should read Safe to Deploy based on your Mongo image upload.
  4. Click on View the vulnerability report for this image. This will bring you to the assessment details page with two tabs:Vulnerable Packages and Policy Violations.
  5. Click on Manage your org’s policies. Here you are presented with two boxes – Deployment Settings for Containers and Image Deployment Impact
  6. Return to the Bluemix Catalog and review the vulnerability assessment for the Let’s Chat image. You can do this by clicking on the purple lets-chat icon and viewing the same vulnerability information on the right hand side of the page.

You have reviewed your pushed images, which were sourced from a public repository, and can now safely deploy them on your hosted Bluemix account. This is a key step in making sure you are running the code which you expect to be running and you are not opening your organization up to security issues, at the expense of agility. You still want to stay secure, even when moving at light-speed!

Push images to Bluemix

It is time to tag them for use in IBM Containers on Bluemix. To do so, you will need to tag them with the repository name you created when you setup your Bluemix account for IBM Container usage.

  • Log in to Bluemix

     $ cf login
     API endpoint: https://api.ng.bluemix.net
    
     Email> jessewei@tw.ibm.com
    
     Password>
     Authenticating...
     OK
    Targeted org jessewei@tw.ibm.com
    
    Select a space (or press enter to skip):
    1. BAO-TW
    2. Scheduler
    
    Space> 1
    Targeted space BAO-TW
     
  • Log into the IBM Containers service

     $ cf ic login
    Deleting the old configuration file...
    Retrieving client certificates from IBM Containers...
    ....
      Example Usage:
      docker ps
      docker images
    
  • First tag your MongoDB image.

  • Remember to use your namespace from the first command below to replace[NAMESPACE] in the tag and push commands below.List your images:
    $ docker images
    REPOSITORY                                               TAG
    mongo                                                    latest
    sdelements/lets-chat                                     latest
    

    Make note of your namespace:

    $ cf ic namespace get
    web1
    

    Tag your Mongo image in a Bluemix-compatible format:

    $ docker tag mongo registry.ng.bluemix.net/web1/mongo
    

    List your images again, now showing the newly tagged image:

    $ docker images
    REPOSITORY                                               TAG
    mongo                                                    latest
    registry.ng.bluemix.net/web1/mongo                       latest
    sdelements/lets-chat                                     latest
    

    The IMAGE ID stays the same and allows us to reuse the existing container image as-is.

  • Next, tag your Let’s Chat image.

  • Push commands below.Tag your Let’s Chat image in a Bluemix-compatible format:
    $ docker tag sdelements/lets-chat registry.ng.bluemix.net/web1/lets-chat
    

    List your images again, now showing the newly tagged image:

    $ docker images
    REPOSITORY                                               TAG
    ...
    
  • Wrap  Let’s Chat image with a simple Dockerfile

    To ensure network connectivity, create a new directory called wrapper

    $ mkdir wrapper
    

    Switch to that directory and run the following command to create a Dockerfile

    $ cd wrapper
    $ echo "FROM sdelements/lets-chat:latest" > Dockerfile
    $ echo "CMD (sleep 60; npm start)" >> Dockerfile
    

    This will create a new Dockerfile that we can build a temporary image from.

    $ docker build -t registry.ng.bluemix.net/web1/lets-chat .
    

    Use this image below to push to Bluemix instead of the base lets-chat image.

  • Push to private registry on Bluemix.

    This allows the IBM Container service to run your container images on the cloud.

    Push your Mongo image to your Bluemix registry:

    $ docker push registry.ng.bluemix.net/web1/mongo
    The push refers to a repository [registry.ng.bluemix.net/ibm_containers_demo_eu/mongo] (len: 1)
    Sending image list
    Pushing repository registry.ng.bluemix.net/web1/mongo (1 tags)
    ...
    Pushing tag for rev [202e2c1fe066] on {https://registry.ng.bluemix.net/v1/repositories/web1/mongo/tags/latest}
    

    Push your Let’s Chat image to your Bluemix registry:

    $ docker push registry.ng.bluemix.net/web1/lets-chat
    The push refers to a repository [registry.ng.bluemix.net/web1/lets-chat] (len: 1)
    Sending image list
    ...
    Pushing tag for rev [2409eb7b9e8c] on {https://registry.ng.bluemix.net/v1/repositories/web1/lets-chat/tags/latest}
    

    Now your images are up in the cloud, in your hosted registry, and ready to run on Bluemix! But first, take a moment to understand what is inside the images you just pushed!

Pull public images

This lab work with two public images, Let’s Chat and MongoDB. First, you will need to pull them down locally before you can tag and push them to your private Bluemix registry.

  Pull the MongoDB image

  • $ docker pull mongo
    Using default tag: latest
    latest: Pulling from library/mongo
    ...
    Digest: sha256:223d59692269be18696be5c4f48e3d4117....
    Status: Downloaded newer image for mongo:latest

    Pull the Let’s Chat image

    $ docker pull sdelements/lets-chat
    Using default tag: latest
    latest: Pulling from sdelements/lets-chat
    ...
    Digest: sha256:98d1637b93a1fcc493bb00bb122602036b7....
    Status: Image is up to date for sdelements/lets-chat:latest
    

    Verify image by running locally

    Start a Mongo instance:

    $ docker run -d --name lc-mongo mongo  
    6ef19c325f6fda8f5c0277337dd797d4e31113daa7da92fbe85fe70557bfcb49
    

    Start a Let’s Chat instance:

    $ docker run -d --name lets-chat --link lc-mongo:mongo -p 8080:8080 sdelements/lets-chat
    4180a983e329947196e317563037bfd0da093ab89add16911...
    

    Access through browser

    In your browser, access http://localhost:8080.

    Stop local containers

    After verify images, these container are not required in following lab.

    Test Stop the containers:

    $ docker stop lets-chat lc-mongo
    lets-chat
    lc-mongo
    

    Delete the containers:

    $ docker rm lets-chat lc-mongo
    lets-chat
    lc-mongo
    

    Congratulations, you’ve pulled and run your first Docker-based web app. Now you will prepare the images to run them on the IBM Containers service in the cloud.

Post Navigation