kafka

1. offset explorer

2. topics, partitions and offset

• data is kept for a limit time (default 1 week)
• data is randomly assigned to partition unless key is provided

3. brokers and topics

• brokers are servers, if you connect to one broker, you connect to all
• a broker store topic.partition, e.g. broker1 can store topic1.partition1 and topic2.partition3
• topic has replication factor > 1, meaning redundant in storage

4. consumer group

• each consumer in group read 1 or more partition, and exclusively

reference

1. apache kafka in 5 minutes 

k8

1. cluster

• a set of nodes
• one master node and many worker nodes

2. ingress/svc/pod

• ingress: route external traffic into cluster, rule based
• svc: like a load balancer
• pod by default is internal within cluster, need to expose as svc to outside

• kubectl create deployment hello-node --image=k8s.gcr.io/echoserver:1.4
• kubectl expose deployment hello-node --type=LoadBalancer --port=8080

3. deployment: blueprint to create pod

• cluster contains nodes (worker) and control plane
• node contains pods

4. kubectl

• similar to aws cli

5. context

• a context contains a cluster, a user and a namespace
• all kubectl cmd runs against current context/cluster

6. manifest yaml/helm/helm charts

• manifest yaml is used to create/modify/delete pod/deployment/svc/ingress
• helm is package manager for k8, can install package like yum/apt/brew

7. kubectl commands

• kubectl config view                                               //show all clusters and contexts
• kubectl config get-contexts                                   //use this instead of above
• kubectl config use-context <context-name>         //switch context
• kubectl config current-context

• kubectl config set-context --current --namespace=<namespace>     //switch namespace
• kubectl config view --minify | grep namespace    //get current namespace

• kubectl config unset users.user1                           //delete user from config
• kubectl config unset contexts.context1                 //delete context from config
• kubectl config unset clusters.cluster1                   //delete cluster

• kubectl get pods 
• kubectl describe pod xxx   
• kubectl logs pod_name                                         //display std log of pod
• kubectl exec pod_name -- env                              //exec env cmd on pod
• kubectl exec -ti pod_name -- bash

• kubectl get svc
• kubectl describe svc xxx

• kubectl get nodes -o wide                                     //all nodes in cluster   

• kubectl get deployments
• kubectl describe deployment xxx

8. kubectx/kubens tool

• kubectx                       //show all contexts
• kubectx ctx1               //switch context

• kubens                        //show all ns
• kubens ns1                 //switch ns

reference

1. boosting your kubectl productivity

2. Learn Kubernetes Basics

test automation

1. read from csv file, use jackson to convert to dto

2. use hardcode expected test data result instead of writing code in the same logic as developer

intellij

 1. java/groovy

• file -> project structure -> project: select sdk
• file -> project structure -> modules: dependencies -> scope (compile: required to build, test, and run a project (the default scope).)
• maven: intellij has its own maven, if using command line maven, need to configure it it intellij -> preferences -> search 'maven' -> maven home path -> select command line maven
• 'maven' tab on right hand side of UI, under lifecycle,  has clean/validate/.. etc
• after performing a lifecycle say install, need to click 'refresh' icon or popup, sometimes it's out-of-sync and not showing 'dependencies' that's downloaded
• settings.xml: can be company customised

2. debug

• drop frame: step过头，跳回上一级函数，重新再来

3. read code

• navigate -> call hierarchy

4. auto generate code

• right click -> generate              //constructor, getter & setter (after define the variable)
• shift + command + T                //generate or update unit test

5. debug multi thread 

• suspend policy 'thread'                                          //link 3
• selective debugging by thread name                     //link 4
• 'monitor' status: blocked, waiting on a lock
• show 'monitor' status when deadlock happens     //link 6

6. reload all maven project button very important

• it's the refresh button in maven view

 

reference

1. IntelliJ debugging tricks

2. intelliJ IDEA | full course | 2020

3. switching between threads in intellij

4. explore intelliJ idea tools to debug multithreaded java apps

5. java thread status - what does "monitor" status mean?

6. show lock/monitor situation in IntelliJ when deadlock occurs

spock and groovy

 

1. spock configuration

• SpockConfig.groovy

2. define annotation in groovy file

@Target([ElementType.TYPE, ElementType.METHOD])

@Retention(RetentionPolicy.RUNTIME)

@interface myTag {

}

reference

1. spock include/exclude specification based on annotation