Tuesday, 27 November 2018

microservice


1. api gateway/nginx

  • routes requests to the appropriate microservice
  • invoke multiple microservices and aggregate the results
  • translate between web protocols such as HTTP and web‑unfriendly protocols that are used internally

reference
1. service discovery

Monday, 22 October 2018

use mac

1.short cuts
  • command + option + ESC (ctrl+alt+del)
  • command + DEL (move to trash)
  • command + tab (cycle apps)
  • command + M (minimize current app)
  • command + W (close window)
  • command + shift + 4 or 3 (take screenshot)
  • command + space (spotlight)
  • command + up/down (go to parent/child dir in finder)
  • command + C / command + option + V (cut and paste file)
  • command + ~ (switch between windows of same application)
  • double click (move the relative position of multiple full screen application)
  • double finger swipe left/right (switch between tabs of browser)
  • drag finder file to terminal (copy its path to terminal)
  • command + shift + G (in finder go to folder)
  • right click + option (copy path)
  • right-click -> service -> new terminal at folder (open terminal at finder)
  • command + shift + T (in textedit remove formatting)
  • view -> show path bar
  • view -> show status bar
  • trackpad -> tap to click
  • track -> secondary click -> click in bottom right corner
  • spread thumb and three finger (show desktop)
  • 4 fingers (mission control)
  • close lid (lock screen)

2. intellij
  • set keymap mac OS X 10.5+                  //for better shortcut key
  • command + two fingers                          //zoom font size
  • command + F9 (save and build)
  • command + E                                          //recent files, also show maven project
  • shift + shift                                              //search everything
  • command + G                                         //search next
  • command + shift + F                              //search text in all files
  • command + B                                         //view definition
  • alt + F7                                                    //find usage
  • command + [                                           //navigate back
  • command + shift + backspace                //last edit location
  • fn + cursor left                                        //goto beginning of line
  • fn + cursor right
  • command + option + L                           //auto format
  • shift + command + 8 .                            //toggle column mode
  • shift + command + F8                            //show all breakpoints
  • option + F8                                             //evaluate expression
  • command + F7                                       //smart step into
  • command + space ()
  • don't forget intellij main menu
  • click .pom file to import maven project, search project recursively
  • eclipse workspace is intellij project, eclipse project is intellij module 
  • select autoscrolling to sources in project setting
  • analyze -> analyze data flow to here
  • debug: can use package.json, "node $NODE_DEBUG_OPTION index.js"
  • debug: can use run configuration 'javascript file' - 'index.js', $NODE_DEBUG_OPTION will be added when clicking debug button
  • the name 'step_definitions' has to be used for cucumber step definition folder so that Intellij can find it from feature file
  • https://www.youtube.com/watch?v=pt3uB0sd5kY

3. vscode 
  • command + P                                              //search file by name
  • ctrl+shift+f                                                  //search all
  • f5                                                                 //debug
  • ctrl+f5                                                         //run
  • ctrl+p                                                          //open recent files
  • ctrl + -                                                        //navigate back
  • ctrl + shift + -                                             //navigate forth

4. brew
  • brew uninstall --force nvm
  • brew list [software]                                   //list all(software) packages installed


reference
1. create intellij run configuration from package.json 'script'
2. intellij can not find step definition from feature file
3. share intellij configuration between projects

Friday, 27 July 2018

json & xml

1. json concept
  • number, string, object, array, boolean, null
  • if start with [, it is a json array
  • if start with {, it is a json object
  • JSON.stringify(obj) convert json object to string
  • JSON.parse(string) convert json string to object
  • json is not extensible

2. json object
myObj = { "name":"John", "age":30, "car":null };

myObj = {                          //nested json object
    "name":"John",
    "cars": {
        "car3":"Fiat"
    }
 }


3. json array (of object)
myObj = {
    "name":"John",
    "cars": [
        { "name":"Ford"},
        { "name":"BMW"},
        { "name":"Fiat"}
    ]
 }


4. java json library (objectmapper)
  • jackson (default for resttemplate, restassured etc)
  • jayway jsonpath

5. xml concept
  • prolog, tag, attribute and element
  • DOM tree, root, parent, child, sibling
  • attribute value must be quoted
  • < less than becomes &lt;
  • online xpath editor @http://xmlgrid.net/xpath.html
  • xml is extensible
  • javascript xml parser
      <?xml version="1.0" encoding="UTF-8"?>       //prolog
      <book category="children">                               //book is tag, category is attribute
          <title>harry Potter</title>                               //element
          <year>2005</year>
      </book>

      parser = new DOMParser();
      xmlDoc = parser.parseFromString(text,"text/xml");


6. xml namespace
<root>

<h:table xmlns:h="http://www.w3.org/TR/html4/">
  <h:tr>
    <h:td>Apples</h:td>
    <h:td>Bananas</h:td>
  </h:tr>
</h:table>

<f:table xmlns:f="https://www.w3schools.com/furniture">
  <f:name>African Coffee Table</f:name>
  <f:width>80</f:width>
  <f:length>120</f:length>
</f:table>

</root>

<table xmlns="http://www.w3.org/TR/html4/">          //default namespace
  <tr>
    <td>Apples</td>
    <td>Bananas</td>
  </tr>
</table>


7. xpath
  • /                         //select from root
  • //                        //from current node, wherever they are
  • .                         //current node
  • ..                        //parent
  • @                      //select attribute

8. java xml library
  • DOM api (tree model, in memory)
  • SAX api (stream based, event driven, push-parsing model)
  • Stax api (stream based, event driven, pull-parsing model)
  • JAXB api (schema based)

9. jaxb
  • marshall: convert java object to xml
  • unmarshall: convert xml to java object
  • jaxb validation: verify that xml or java object tree meet all constraint in xml schema
  • convert schema to jaxb class in eclipse (right click)
  • convert schema to jaxb class online ()

10. xml schema
<?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:element name="note">
  <xs:complexType>
    <xs:sequence>
      <xs:element name="to" type="xs:string"/>
    </xs:sequence>
  </xs:complexType>
</xs:element>

</xs:schema>

Saturday, 30 June 2018

docker

1. docker container, virtual machine, hypervisor and docker engine
  • docker container runs on docker engine, vm runs on hypervisor (vm engine)
  • docker container is a running instance of docker image

2. docker command
  • docker --version
  • docker ps                                                          //list running containers
  • docker pull
  • docker image ls -a
  • docker rmi $(docker images -q)                        //remove all images
  • docker run -d -t                                                  //detached, run in background
  • docker run -it                                                     //run in foreground with shell
  • docker run -p 127.0.0.1:80:8080/tcp                 //public host:container port
  • docker run -it --link server:redis --name client busybox    //link client to server, add server in client's host file, can ping server from client
  • docker stop $(docker ps -aq)                             //stop all containers
  • docker rm $(docker ps -aq)                               //remove all containers
  • docker container ls -a                                        //same as docker ps -a
  • docker container start
  • docker container stop
  • docker container rm
  • docker build -t <mytag> .                                //build image from docker file
  • docker run -d mytag                                        //run the image
  • docker run -it mytag /bin/ash                          //run image with shell
  • docker run -it <mytag> sh                               //get the content of image
  • docker export $(docker ps -lq) | tar tf -
  • docker cp                                                        //copy file from local machine to container

3. dockerfile / docker-compose.yml / docker image / docker container
  • docker container is running instance of docker image
  • dockerfile is a file that produces a docker image when you build it
  • dockerfile create a single image that can be run as a container
  • dockerfile normally define the 'static' environment
  • docker-compose.yml can build multiple container in one compose file that works together as an application
  • docker-compose.yml file normally define the 'dynamic' services
  • docker-compose.yml can overwrite settings in dockerfile and docker hub

4. docker-compose
  • build: go through services listed in docker-compose.yml, look for 'build' key, and build image based on dockerfile
  • up vs start: 'up' will build, create and start entire application for the 1st time, 'start' will start the containers that are stopped by 'stop'
  • down vs stop: 'down' will stop and remove entire application, 'stop' will stop the containers
  • looks like all we need is up/down and start/stop commands

5. sample dockerfile

      # base image
      FROM python:3.6

      # RUN command
      RUN mkdir -p /usr/src/app

      # make the folder as a working directory
      WORKDIR /usr/src/app                                   //set working dir for any following RUN
                                                                               //COMMAND, ENTRYPOINT, COPY, AND

      # copy host files to the image folder
      COPY package.json .                                       //package.json is host,  . is container

      # CMD command
      CMD [ “npm”, “start” ]


6. sample docker compose file

      version: '2'
      services:

        selenium_hub:
          image: selenium/hub
          ports:
            - "4444"

        app:
          build: app/.                                                       //where dockerfile is
          ports:
            - "80"

        robottests:
          command: testui/wait-for-it.sh -t 15 chromenode:5555 
          depends_on:
            - selenium_hub
            - app
          build: testui/.                                                  //where dockerfile is
          volumes:
            - {WORKSPACE}/reports:/frontend-integration-tests/reports .   //map host to container

          //{WORKSPACE} is a Jenkins environment variable that defines a root of a repository with the project files


7. sample jenkins file that use docker

      node {
          stage('build an image with Tests') {
              sh """
                docker-compose build robottests:{BUILD_NUMBER}
              """
          }

          stage('run Docker Compose') {
              sh """#!/bin/bash -e
                docker-compose run --rm robottests:{BUILD_NUMBER}
              """
          }

          stage('stop all containers') {
              sh """
                docker-compose down
              """
          }
      }

8. link docker container
  • link to another container by name so that you can ping it (add to hostname)

9. swarm mode
  • like selenium grid, docker host can be manager and worker

10. docker volume
  • to persist data
  • volume will persist directory in container to host directory
  • host directory can be specified or anonymous
  • use 'docker inspect container_name' to see volume info under 'Mounts' section
  • docker volume ls                                    //list all volumes in the host

11. data sharing in docker
  • dockerfile copy & add: add can do more than copy
  • bind mount: /path/on/host:/path/in/container
  • volume: volumes are managed by docker and are independent of the host filesystem, my_volume:/path/in/container
  • share data between containers: use bind mount or volume

12. docker tag
  • docker build -t repository_name/image_name:tag_name .   //add tag to the image

13. running test automation in docker (selenium example)
  • create docker image that contain everything necessary to run test (java, mvn, content of test)
  • run image as docker container
  • run test within container against external selenium grid
  • after the test was run, temporary container would be destroyed (but persistent test result should be archived to S3 bucket)

14. docker registry and docker repositories
  • default registry is dockerhub
  • a registry is organised into repositories, each repository holds all versions of an image

15. ubank docker commands
  • docker login ubank-docker.artifactory.dss.ext.national.com.au -u **** -p ****
  • docker build -t mytag -f MyDockerfile .
  • docker run -it mytag /bin/bash


reference

Sunday, 3 June 2018

data structure

1. tree
  • pre-order: root node pre child
  • post-order: root node post child
  • in-order: root node inbetween child 
  • always use stack to iterate

2. pre-order
  • push root, pop root
  • push right child, push left child
  • top of stack become new root
  • go back

3. post-order
  • push root
  • push left child
  • top of stack become new root
  • if no left child, pop
  • right child become new root
4. binary search tree (balanced)
  • if not balanced, bst may end up as linked list
  • avl tree and red-black tree are bst

5. other type of tree
  • heap (insert from left to right)
  • priority queue (insert from left to right)
  • trie (prefix tree that often store string)

reference

Wednesday, 25 April 2018

node

1. node concepts
  • http server + request router + request handler
  • use DI everywhere, pass router/handler as parameter to server
  • no 'return' is used, use async function, 'response' object is passed all the way to handler
  • in node, any async function accepts a callback as the last parameter
  • in node, any callback function accepts an error as the first parameter

2. even loop
  • timer: execute timer callbacks
  • pending callbacks: execute I/O callbacks
  • poll: incoming I/O events and callbacks

3. npm config file
  • ~/.npmrc

4. module.export and require
  • module.export return an object
  • require use the object
      module.exports = function(app, db) {};  // file1.js
      require('./app/routes')(app, {});               // file2.js

5. node/nvm commands
  • node -v
  • nvm --version
  • nvm ls-remote
  • nvm install x.x.x
  • nvm uninstall x.x.x
  • nvm use x.x.x
  • nvm use node                                   //switch to latest
  • nvm list
  • nvm install --lts                                //install latest
  • npm install -d                                   //--loglevel info
  • npm install jsonpath --save-dev      //save it in package.json
  • npm install -D jsonpath                   //-D is same as --save-dev
  • npm init                                           //new node app, generate package.json
  • node hello.js

6. node app config file
  • ./config by default
  • can be overwritten: process.env["NODE_CONFIG_DIR"] = __dirname + "/configDir/";

7. commonly lodash functions
  • .find()

8. returning everything as objects of functions

const retryCounter = () => {
    let retryCounter = 0
    const totalRetry = 10 //config.get('errorHandler.retryTotal')

    return {
        retryTotal: () => {
            return totalRetry
        },
        count: () => {
            return retryCounter
        },
        add: () => {
            retryCounter += 1
        }
    }
}

9. print stack trace
  • console.trace("Here I am!")

10. package-lock.json (normally more important than package.json, but)
  • package.json overrule the package-lock.json if package.json has been updated
  • 'npm install' sometimes does not update package-lock.json, better to use 'npm update' 
  • version in package.json
major.minor.patch
1.0.2
~1.0.2 means to install version 1.0.2 or the latest patch version such as 1.0.4
^1.0.2 means to install version 1.0.2 or the latest minor or patch version such as 1.1.0


11. dependencies vs devDependencies
  • dependencies is installed when using 'npm install' or 'npm install $package'
  • devDependencies is installed when using 'npm install' or 'npm install --save-dev $package'
  • dependencies is needed to run a package
  • devDependencies is need to develop a package



reference

Saturday, 7 April 2018

maven

1. lifecycle
  • clean
  • default
  • site
  • packaging of project: if no <packaging> value found, jar is used.

2. phase
  • clean (3)
  • default (21)
  • site (4)
  • 'mvn install' will execute all phases preceding 'install'

3. phase and plugin:goal
  • phase rely on plugin:goal to carryout the task
  • default binding between phase and goal exist:  test | surefire:test
  • 'mvn test' will execute default plugin/goal, i.e. 'mvn surefire:test'
  • plugin/goal can be executed outside of phase, e.g. 'mvn clean dependency:copy-dependencies package'
  • when execute maven you can specify 'phase' or 'goal'
  • if you specify 'phase', it will run all phases up till the specified phase, and it will run all goals attached to each phase
  • if you specify a 'goal', it will run all phases up till the phase for the goal, then it will run that goal
  • mvn-plugins-package-goal

4. mvn profile
  • use profile to customize build for different env
  • use 'mvn -P' option in command line to activate profile
  • profile can also be activated based on OS variable or jdk version

5. mvn command
  • mvn clean //remove target dir
  • mvn test //execute unit test
  • mvn install //install artifact to local repository
  • mvn install -DskipTests //skip test

6. maven surefire and failsafe plugin
  • surefire plugin is used in 'test' phase, it's designed to run unit tests and if any of the tests fail then it will fail the build immediately
  • surefire plugin has one goal: 'test'
  • failsafe plugin is used in 'integration-tests' and 'verify' phase, it's designed to run integration tests, and do not fail the build if there are test failures
  • failsafe plugin has two goals: 'integration-test' runs the integration tests of an application; 'verify' verifies that the integration tests of an application passed

7. parallel/fork
  • run junit in parallel two options

reference

Sunday, 4 March 2018

proxy


1. windows wininet proxy
  • configured in 'LAN settings' (in below order of precedence)
  • 'automatically detect settings'
  • 'use automatic configuration script'
  • 'proxy server'
  • unset $http_proxy                              //unset proxy

2. 'automatically detect settings'
  • If the client is able to automatically detect a proxy script's URL, it will then attempt to download and use that proxy configuration script

3. 'use automatic configuration script'
  • example: http://proxy.contoso.com/proxy.pac

4. 'proxy server'

5. mac system proxy

6. fiddler
  • it use wininet proxy
  • it changes wininet proxy on fiddler start, and restores it on fiddler exit

7. reverse proxy server

8. mock server
  • hoverfly





reference
1. understanding web proxy configuration
2. fiddler in action
3. reverse proxy server
4. how to configure proxy server on mac