BogoToBogo
  • Home
  • About
  • Big Data
  • Machine Learning
  • AngularJS
  • Python
  • C++
  • go
  • DevOps
  • Kubernetes
  • Algorithms
  • More...
    • Qt 5
    • Linux
    • FFmpeg
    • Matlab
    • Django 1.8
    • Ruby On Rails
    • HTML5 & CSS

Setting Up Remote Git Repository and Fetch / Pull - 2020





Bookmark and Share





bogotobogo.com site search:

Setting up Remote Git

In this chapter, we'll setup remote git repository and play with local repository

To setup the remote git repo:

$ ssh account@bogotobogo.com
...
$ mkdir MyProject.git
$ cd MyProject.git
$ git --bare init

Here is the structure of the git we've just made:

$ tree -a
treeA

Now, we'll set up two git repos at local machines, A and B. A will be connected to remote repo via git remote add while B will be cloned from remote repo via git clone.



Setting up Local (A) Git

On local machine (A):

$ mkdir MyProject
$ cd MyProject
$ echo "file1" > file1
$ git init 
$ git add *
$ tree -a
treeB

To check current status:

$ git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
#   (use "git rm --cached ..." to unstage)
#
#	new file:   file1
#

If we commit the change:

$ git commit -m "My initial commit message"
[master (root-commit) 1c96ab9] My initial commit message
 1 file changed, 1 insertion(+)
 create mode 100644 file1
$ git status
# On branch master
nothing to commit, working directory clean

Note that the change is limited to local. Now, we may want to make the change to the remote.




Pushing local repo (A) to remote Git

The git is a distributed version control system. Most operations are limited to local. To communicate with the outside world, git uses what is called remote. It is a repository other than the one on our local machine which we can push our changes into (so that other people can see them) or pull from (so that we can get others changes).

$ git remote add origin account@bogotobogo.com:MyProject.git

We just created an entry in our git config that specifies a name for a particular URL via git remote add. Note that we must have an existing git repo to use this. As we'll see later, unlike the git remote add, the git clone creates a new git repository by copying an existing one located at the URI we specify.

To share the commit we've done, we need to push our changes to the remote repository. To do this, we run git push [alias] [branch] which will attempt to make our [branch] the new [branch] on the [alias] remote.

Since we've already setup the remote alias via git remote add origin URI, in our push command, we can just push to origin instead of typing out the whole URI.

Let's try it by initially pushing our master branch to the new origin remote we created earlier:

The syntax is:

git push [<remoteName> <branchName>]

The actual command looks like this:

$ git push origin master
$ tree -a
treeC

The git push origin master command says "push the commits in the local branch named master to the remote named origin". Once this is executed, all the stuff that we last synchronised with origin will be sent to the remote repository and other people will be able to see them there.

The changes been made to the remote repository by the push command is shown in the picture below:

$ tree -a
treeD


Cloning the remote Git at Local (B)

Now, another team member can share the file1 by cloning the master branch of the remote repository at local B:

$ git clone account@bogotobogo.com:MyProject.git
$ cd MyProject
$ ls
file1



Showing the remote Git

To see which remote servers we have configured, we can run the git remote command. It lists the aliases of each remote handle we've specified. If we've cloned our repository, we should at least see origin - that is the default name Git gives to the server we cloned from.

$ git remote -v
origin	account@bogotobogo.com:MyProject.git (fetch)
origin	account@bogotobogo.com:MyProject.git (push)



Push / Pull and Fetch

Another team member at local (B) updates the remote using push:

$ echo "file2" > file2
$ git add file2
$ git commit -m "file2 commit"
$ git push origin master

Now, I (at local A) can get the updates using pull:

$ git pull origin
account@bogotobogo.com's password: 
You asked to pull from the remote 'origin', but did not specify
a branch. Because this is not the default configured remote
for your current branch, you must specify a branch on the command line.

$ git pull origin master
account@bogotobogo.com's password: 
From bogotobogo.com:MyProject
 * branch            master     -> FETCH_HEAD
Updating 1c96ab9..fb8483b
Fast-forward
 file2 | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 file2

$ ls
file1  file2

A fetch command does not make any changes to local branches, so we will need to merge a remote branch with a paired local branch to incorporate newly fetch changes. In other words, the fetch alone cannot do update. We need to use merge command as well as shown below. Suppose, the remote has been updated by adding file3.

$ git fetch origin
account@bogotobogo.com's password: 
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From bogotobogo.com:MyProject
   fb8483b..9357619  master     -> origin/master

$ ls
file1  file2

$ git merge origin/master
Updating fb8483b..9357619
Fast-forward
 file3 | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 file3

$ ls
file1  file2  file3

We first ran git fetch origin to tell Git to fetch down all the data it has that we do not, then we ran git merge origin/master to merge into our current branch anything new we see on the server.






Git diff

Suppose we made a change to a file, file4 at Local A:

$ cat file4
file4
midified by local A
git_diff.png

Picture from How do I show the changes which have been staged?.

We can see the difference between the file at the working directory and at the staging area:

$ git diff
diff --git a/file4 b/file4
index bfd6a65..ff56b6b 100644
--- a/file4
+++ b/file4
@@ -1 +1,2 @@
 file4
+midified by local A

But once we put the file into the staging area, we do not see the difference between working tree and staging area:

$ git add file4
$ git diff

However, we still can see the difference between staging area and HEAD:

$ git diff --cached
diff --git a/file4 b/file4
index bfd6a65..ff56b6b 100644
--- a/file4
+++ b/file4
@@ -1 +1,2 @@
 file4
+midified by local A

After a commit, we can't see the difference between staging area and HEAD:

$ git commit -m "commit modified file4 by local A"
[master 7a6cd63] commit modified file4 by local A
 1 file changed, 1 insertion(+)
$ git diff --cached

Now we do fecth and merge:

$ git fetch origin
$ git merge origin/master
Auto-merging file4
CONFLICT (content): Merge conflict in file4
Automatic merge failed; fix conflicts and then commit the result.



Git conflicts and merge

In this section we'll see how we can resolve conflicts during fetch, pull, and merge.

If we want to update local B from remote, we do:

$ git fetch origin
$ git merge origin/master
Updating 25a70af..7a6cd63
Fast-forward
 file4 | 1 +
 1 file changed, 1 insertion(+)

$ cat file4
file4
midified by local A

This time, file4 is pushed to the remote by local A:

$ cat file4
file4
midified by local A
midified AGAIN by local A

The local B also updated it but from the old file4:

$ cat file4
file4
midified by local A
midified by local B

git add file4
$ git diff
$ git commit -m "commit by local B"
[master 3a7a932] commit by local B
 1 file changed, 1 insertion(+)

git diff HEAD

Now we do fecth and merge:

$ git fetch origin
$ git merge origin/master
Auto-merging file4
CONFLICT (content): Merge conflict in file4
Automatic merge failed; fix conflicts and then commit the result.

$ git diff --cached
* Unmerged path file4

$ git diff HEAD
diff --git a/file4 b/file4
index 0a86872..19d635a 100644
--- a/file4
+++ b/file4
@@ -1,3 +1,7 @@
 file4
 midified by local A
+<<<<<<< HEAD
 midified by local B
+=======
+midified AGAIN by local A
+>>>>>>> origin/master

Now that we see the conflicts, we need to updates the file4 from the remote and then modify it.

$ cat file4
file4
midified by local A
midified AGAIN by local A

$ git merge origin/master
error: 'merge' is not possible because you have unmerged files.
hint: Fix them up in the work tree,
hint: and then use 'git add/rm ' as
hint: appropriate to mark resolution and make a commit,
hint: or use 'git commit -a'.
fatal: Exiting because of an unresolved conflict.

$ git add file4
$ git commit -m "resolving file4 conflicts local B"
[master 389bd9e] resolving file4 conflicts local B

$ git merge origin/master
Already up-to-date.

Now we can update the file4 and push it to remote:

$ cat file4
file4
midified by local A
midified AGAIN by local A
midified by local B

$ git add file4
$ git commit -m "midified by local B"

As we've done before, the file4 can be pulled from remote to local A:

$ git fetch origin
$ git merge origin/master
Updating ec81df2..dc017c2
Fast-forward
 file4 | 1 +
 1 file changed, 1 insertion(+)

We can see it's been updated:

$ cat file4
file4
midified by local A
midified AGAIN by local A
midified by local B

We can use the mergetool such as kdiff3:

kdiff3.png

Here are the terms used for merging:

  1. base: name of a temporary file containing the common base for the merge.
  2. local: name of a temporary file containing the contents of the file on the current branch.
  3. remote: name of a temporary file containing the contents of the file to be merged.


Git/GitHub Tutorial

  • One page express tutorial for GIT and GitHub
  • Installation
  • add/status/log
  • commit and diff
  • git commit --amend
  • Deleting and Renaming files
  • Undoing Things : File Checkout & Unstaging
  • Reverting commit
  • Soft Reset - (git reset --soft <SHA key>)
  • Mixed Reset - Default
  • Hard Reset - (git reset --hard <SHA key>)
  • Creating & switching Branches
  • Fast-forward merge
  • Rebase & Three-way merge
  • Merge conflicts with a simple example
  • GitHub Account and SSH
  • Uploading to GitHub
  • GUI
  • Branching & Merging
  • Merging conflicts
  • GIT on Ubuntu and OS X - Focused on Branching
  • Setting up a remote repository / pushing local project and cloning the remote repo
  • Fork vs Clone, Origin vs Upstream
  • Git/GitHub Terminologies
  • Git/GitHub via SourceTree II : Branching & Merging
  • Git/GitHub via SourceTree III : Git Work Flow
  • Git/GitHub via SourceTree IV : Git Reset
  • Git wiki - quick command reference











  • Ph.D. / Golden Gate Ave, San Francisco / Seoul National Univ / Carnegie Mellon / UC Berkeley / DevOps / Deep Learning / Visualization

    YouTubeMy YouTube channel

    Sponsor Open Source development activities and free contents for everyone.

    Thank you.

    - K Hong





    Git/GitHub Tutorial



    One page express tutorial for GIT and GitHub

    Installation

    add/status/log

    commit and diff

    git commit --amend

    Deleting and Renaming files

    Undoing Things : File Checkout & Unstaging

    Reverting commit

    Soft Reset - (git reset --soft <SHA key>)

    Mixed Reset - Default

    Hard Reset - (git reset --hard <SHA key>)

    Creating & switching Branches

    Fast-forward merge

    Rebase & Three-way merge

    Merge conflicts with a simple example

    GitHub Account and SSH

    Uploading to GitHub

    GUI

    Branching & Merging

    Merging conflicts

    GIT on Ubuntu and OS X - Focused on Branching

    Setting up a remote repository / pushing local project and cloning the remote repo

    Fork vs Clone, Origin vs Upstream

    Git/GitHub Terminologies

    Git/GitHub via SourceTree II : Branching & Merging

    Git/GitHub via SourceTree III : Git Work Flow

    Git/GitHub via SourceTree IV : Git Reset

    Git wiki - quick command reference



    Sponsor Open Source development activities and free contents for everyone.

    Thank you.

    - K Hong







    C++ Tutorials

    C++ Home

    Algorithms & Data Structures in C++ ...

    Application (UI) - using Windows Forms (Visual Studio 2013/2012)

    auto_ptr

    Binary Tree Example Code

    Blackjack with Qt

    Boost - shared_ptr, weak_ptr, mpl, lambda, etc.

    Boost.Asio (Socket Programming - Asynchronous TCP/IP)...

    Classes and Structs

    Constructor

    C++11(C++0x): rvalue references, move constructor, and lambda, etc.

    C++ API Testing

    C++ Keywords - const, volatile, etc.

    Debugging Crash & Memory Leak

    Design Patterns in C++ ...

    Dynamic Cast Operator

    Eclipse CDT / JNI (Java Native Interface) / MinGW

    Embedded Systems Programming I - Introduction

    Embedded Systems Programming II - gcc ARM Toolchain and Simple Code on Ubuntu and Fedora

    Embedded Systems Programming III - Eclipse CDT Plugin for gcc ARM Toolchain

    Exceptions

    Friend Functions and Friend Classes

    fstream: input & output

    Function Overloading

    Functors (Function Objects) I - Introduction

    Functors (Function Objects) II - Converting function to functor

    Functors (Function Objects) - General



    Git and GitHub Express...

    GTest (Google Unit Test) with Visual Studio 2012

    Inheritance & Virtual Inheritance (multiple inheritance)

    Libraries - Static, Shared (Dynamic)

    Linked List Basics

    Linked List Examples

    make & CMake

    make (gnu)

    Memory Allocation

    Multi-Threaded Programming - Terminology - Semaphore, Mutex, Priority Inversion etc.

    Multi-Threaded Programming II - Native Thread for Win32 (A)

    Multi-Threaded Programming II - Native Thread for Win32 (B)

    Multi-Threaded Programming II - Native Thread for Win32 (C)

    Multi-Threaded Programming II - C++ Thread for Win32

    Multi-Threaded Programming III - C/C++ Class Thread for Pthreads

    MultiThreading/Parallel Programming - IPC

    Multi-Threaded Programming with C++11 Part A (start, join(), detach(), and ownership)

    Multi-Threaded Programming with C++11 Part B (Sharing Data - mutex, and race conditions, and deadlock)

    Multithread Debugging

    Object Returning

    Object Slicing and Virtual Table

    OpenCV with C++

    Operator Overloading I

    Operator Overloading II - self assignment

    Pass by Value vs. Pass by Reference

    Pointers

    Pointers II - void pointers & arrays

    Pointers III - pointer to function & multi-dimensional arrays

    Preprocessor - Macro

    Private Inheritance

    Python & C++ with SIP

    (Pseudo)-random numbers in C++

    References for Built-in Types

    Socket - Server & Client

    Socket - Server & Client 2

    Socket - Server & Client 3

    Socket - Server & Client with Qt (Asynchronous / Multithreading / ThreadPool etc.)

    Stack Unwinding

    Standard Template Library (STL) I - Vector & List

    Standard Template Library (STL) II - Maps

    Standard Template Library (STL) II - unordered_map

    Standard Template Library (STL) II - Sets

    Standard Template Library (STL) III - Iterators

    Standard Template Library (STL) IV - Algorithms

    Standard Template Library (STL) V - Function Objects

    Static Variables and Static Class Members

    String

    String II - sstream etc.

    Taste of Assembly

    Templates

    Template Specialization

    Template Specialization - Traits

    Template Implementation & Compiler (.h or .cpp?)

    The this Pointer

    Type Cast Operators

    Upcasting and Downcasting

    Virtual Destructor & boost::shared_ptr

    Virtual Functions



    Programming Questions and Solutions ↓

    Strings and Arrays

    Linked List

    Recursion

    Bit Manipulation

    Small Programs (string, memory functions etc.)

    Math & Probability

    Multithreading

    140 Questions by Google



    Qt 5 EXPRESS...

    Win32 DLL ...

    Articles On C++

    What's new in C++11...

    C++11 Threads EXPRESS...

    Go Tutorial

    OpenCV...





    DevOps



    Phases of Continuous Integration

    Software development methodology

    Introduction to DevOps

    Samples of Continuous Integration (CI) / Continuous Delivery (CD) - Use cases

    Artifact repository and repository management

    Linux - General, shell programming, processes & signals ...

    RabbitMQ...

    MariaDB

    New Relic APM with NodeJS : simple agent setup on AWS instance

    Nagios on CentOS 7 with Nagios Remote Plugin Executor (NRPE)

    Nagios - The industry standard in IT infrastructure monitoring on Ubuntu

    Zabbix 3 install on Ubuntu 14.04 & adding hosts / items / graphs

    Datadog - Monitoring with PagerDuty/HipChat and APM

    Install and Configure Mesos Cluster

    Cassandra on a Single-Node Cluster

    Container Orchestration : Docker Swarm vs Kubernetes vs Apache Mesos

    OpenStack install on Ubuntu 16.04 server - DevStack

    AWS EC2 Container Service (ECS) & EC2 Container Registry (ECR) | Docker Registry

    CI/CD with CircleCI - Heroku deploy

    Introduction to Terraform with AWS elb & nginx

    Docker & Kubernetes

    Kubernetes I - Running Kubernetes Locally via Minikube

    Kubernetes II - kops on AWS

    Kubernetes III - kubeadm on AWS

    AWS : EKS (Elastic Container Service for Kubernetes)

    CI/CD Github actions

    CI/CD Gitlab



    DevOps / Sys Admin Q & A



    (1A) - Linux Commands

    (1B) - Linux Commands

    (2) - Networks

    (2B) - Networks

    (3) - Linux Systems

    (4) - Scripting (Ruby/Shell)

    (5) - Configuration Management

    (6) - AWS VPC setup (public/private subnets with NAT)

    (6B) - AWS VPC Peering

    (7) - Web server

    (8) - Database

    (9) - Linux System / Application Monitoring, Performance Tuning, Profiling Methods & Tools

    (10) - Trouble Shooting: Load, Throughput, Response time and Leaks

    (11) - SSH key pairs, SSL Certificate, and SSL Handshake

    (12) - Why is the database slow?

    (13) - Is my web site down?

    (14) - Is my server down?

    (15) - Why is the server sluggish?

    (16A) - Serving multiple domains using Virtual Hosts - Apache

    (16B) - Serving multiple domains using server block - Nginx

    (16C) - Reverse proxy servers and load balancers - Nginx

    (17) - Linux startup process

    (18) - phpMyAdmin with Nginx virtual host as a subdomain

    (19) - How to SSH login without password?

    (20) - Log Rotation

    (21) - Monitoring Metrics

    (22) - lsof

    (23) - Wireshark introduction

    (24) - User account management

    (25) - Domain Name System (DNS)

    (26) - NGINX SSL/TLS, Caching, and Session

    (27) - Troubleshooting 5xx server errors

    (28) - Linux Systemd: journalctl

    (29) - Linux Systemd: FirewallD

    (30) - Linux: SELinux

    (31) - Linux: Samba

    (0) - Linux Sys Admin's Day to Day tasks





    Jenkins



    Install

    Configuration - Manage Jenkins - security setup

    Adding job and build

    Scheduling jobs

    Managing_plugins

    Git/GitHub plugins, SSH keys configuration, and Fork/Clone

    JDK & Maven setup

    Build configuration for GitHub Java application with Maven

    Build Action for GitHub Java application with Maven - Console Output, Updating Maven

    Commit to changes to GitHub & new test results - Build Failure

    Commit to changes to GitHub & new test results - Successful Build

    Adding code coverage and metrics

    Jenkins on EC2 - creating an EC2 account, ssh to EC2, and install Apache server

    Jenkins on EC2 - setting up Jenkins account, plugins, and Configure System (JAVA_HOME, MAVEN_HOME, notification email)

    Jenkins on EC2 - Creating a Maven project

    Jenkins on EC2 - Configuring GitHub Hook and Notification service to Jenkins server for any changes to the repository

    Jenkins on EC2 - Line Coverage with JaCoCo plugin

    Setting up Master and Slave nodes

    Jenkins Build Pipeline & Dependency Graph Plugins

    Jenkins Build Flow Plugin

    Pipeline Jenkinsfile with Classic / Blue Ocean

    Jenkins Setting up Slave nodes on AWS

    Jenkins Q & A





    Puppet



    Puppet with Amazon AWS I - Puppet accounts

    Puppet with Amazon AWS II (ssh & puppetmaster/puppet install)

    Puppet with Amazon AWS III - Puppet running Hello World

    Puppet Code Basics - Terminology

    Puppet with Amazon AWS on CentOS 7 (I) - Master setup on EC2

    Puppet with Amazon AWS on CentOS 7 (II) - Configuring a Puppet Master Server with Passenger and Apache

    Puppet master /agent ubuntu 14.04 install on EC2 nodes

    Puppet master post install tasks - master's names and certificates setup,

    Puppet agent post install tasks - configure agent, hostnames, and sign request

    EC2 Puppet master/agent basic tasks - main manifest with a file resource/module and immediate execution on an agent node

    Setting up puppet master and agent with simple scripts on EC2 / remote install from desktop

    EC2 Puppet - Install lamp with a manifest ('puppet apply')

    EC2 Puppet - Install lamp with a module

    Puppet variable scope

    Puppet packages, services, and files

    Puppet packages, services, and files II with nginx Puppet templates

    Puppet creating and managing user accounts with SSH access

    Puppet Locking user accounts & deploying sudoers file

    Puppet exec resource

    Puppet classes and modules

    Puppet Forge modules

    Puppet Express

    Puppet Express 2

    Puppet 4 : Changes

    Puppet --configprint

    Puppet with Docker

    Puppet 6.0.2 install on Ubuntu 18.04





    Chef



    What is Chef?

    Chef install on Ubuntu 14.04 - Local Workstation via omnibus installer

    Setting up Hosted Chef server

    VirtualBox via Vagrant with Chef client provision

    Creating and using cookbooks on a VirtualBox node

    Chef server install on Ubuntu 14.04

    Chef workstation setup on EC2 Ubuntu 14.04

    Chef Client Node - Knife Bootstrapping a node on EC2 ubuntu 14.04





    Docker & K8s



    Docker install on Amazon Linux AMI

    Docker install on EC2 Ubuntu 14.04

    Docker container vs Virtual Machine

    Docker install on Ubuntu 14.04

    Docker Hello World Application

    Nginx image - share/copy files, Dockerfile

    Working with Docker images : brief introduction

    Docker image and container via docker commands (search, pull, run, ps, restart, attach, and rm)

    More on docker run command (docker run -it, docker run --rm, etc.)

    Docker Networks - Bridge Driver Network

    Docker Persistent Storage

    File sharing between host and container (docker run -d -p -v)

    Linking containers and volume for datastore

    Dockerfile - Build Docker images automatically I - FROM, MAINTAINER, and build context

    Dockerfile - Build Docker images automatically II - revisiting FROM, MAINTAINER, build context, and caching

    Dockerfile - Build Docker images automatically III - RUN

    Dockerfile - Build Docker images automatically IV - CMD

    Dockerfile - Build Docker images automatically V - WORKDIR, ENV, ADD, and ENTRYPOINT

    Docker - Apache Tomcat

    Docker - NodeJS

    Docker - NodeJS with hostname

    Docker Compose - NodeJS with MongoDB

    Docker - Prometheus and Grafana with Docker-compose

    Docker - StatsD/Graphite/Grafana

    Docker - Deploying a Java EE JBoss/WildFly Application on AWS Elastic Beanstalk Using Docker Containers

    Docker : NodeJS with GCP Kubernetes Engine

    Docker : Jenkins Multibranch Pipeline with Jenkinsfile and Github

    Docker : Jenkins Master and Slave

    Docker - ELK : ElasticSearch, Logstash, and Kibana

    Docker - ELK 7.6 : Elasticsearch on Centos 7 Docker - ELK 7.6 : Filebeat on Centos 7

    Docker - ELK 7.6 : Logstash on Centos 7

    Docker - ELK 7.6 : Kibana on Centos 7 Part 1

    Docker - ELK 7.6 : Kibana on Centos 7 Part 2

    Docker - ELK 7.6 : Elastic Stack with Docker Compose

    Docker - Deploy Elastic Cloud on Kubernetes (ECK) via Elasticsearch operator on minikube

    Docker - Deploy Elastic Stack via Helm on minikube

    Docker Compose - A gentle introduction with WordPress

    Docker Compose - MySQL

    MEAN Stack app on Docker containers : micro services

    Docker Compose - Hashicorp's Vault and Consul Part A (install vault, unsealing, static secrets, and policies)

    Docker Compose - Hashicorp's Vault and Consul Part B (EaaS, dynamic secrets, leases, and revocation)

    Docker Compose - Hashicorp's Vault and Consul Part C (Consul)

    Docker Compose with two containers - Flask REST API service container and an Apache server container

    Docker compose : Nginx reverse proxy with multiple containers

    Docker compose : Nginx reverse proxy with multiple containers

    Docker & Kubernetes : Envoy - Getting started

    Docker & Kubernetes : Envoy - Front Proxy

    Docker & Kubernetes : Ambassador - Envoy API Gateway on Kubernetes

    Docker Packer

    Docker Cheat Sheet

    Docker Q & A

    Kubernetes Q & A - Part I

    Kubernetes Q & A - Part II

    Docker - Run a React app in a docker

    Docker - Run a React app in a docker II (snapshot app with nginx)

    Docker - NodeJS and MySQL app with React in a docker

    Docker - Step by Step NodeJS and MySQL app with React - I

    Installing LAMP via puppet on Docker

    Docker install via Puppet

    Nginx Docker install via Ansible

    Apache Hadoop CDH 5.8 Install with QuickStarts Docker

    Docker - Deploying Flask app to ECS

    Docker Compose - Deploying WordPress to AWS

    Docker - WordPress Deploy to ECS with Docker-Compose (ECS-CLI EC2 type)

    Docker - ECS Fargate

    Docker - AWS ECS service discovery with Flask and Redis

    Docker & Kubernetes: minikube version: v1.31.2, 2023

    Docker & Kubernetes 1 : minikube

    Docker & Kubernetes 2 : minikube Django with Postgres - persistent volume

    Docker & Kubernetes 3 : minikube Django with Redis and Celery

    Docker & Kubernetes 4 : Django with RDS via AWS Kops

    Docker & Kubernetes : Kops on AWS

    Docker & Kubernetes : Ingress controller on AWS with Kops

    Docker & Kubernetes : HashiCorp's Vault and Consul on minikube

    Docker & Kubernetes : HashiCorp's Vault and Consul - Auto-unseal using Transit Secrets Engine

    Docker & Kubernetes : Persistent Volumes & Persistent Volumes Claims - hostPath and annotations

    Docker & Kubernetes : Persistent Volumes - Dynamic volume provisioning

    Docker & Kubernetes : DaemonSet

    Docker & Kubernetes : Secrets

    Docker & Kubernetes : kubectl command

    Docker & Kubernetes : Assign a Kubernetes Pod to a particular node in a Kubernetes cluster

    Docker & Kubernetes : Configure a Pod to Use a ConfigMap

    AWS : EKS (Elastic Container Service for Kubernetes)

    Docker & Kubernetes : Run a React app in a minikube

    Docker & Kubernetes : Minikube install on AWS EC2

    Docker & Kubernetes : Cassandra with a StatefulSet

    Docker & Kubernetes : Terraform and AWS EKS

    Docker & Kubernetes : Pods and Service definitions

    Docker & Kubernetes : Headless service and discovering pods

    Docker & Kubernetes : Service IP and the Service Type

    Docker & Kubernetes : Kubernetes DNS with Pods and Services

    Docker & Kubernetes - Scaling and Updating application

    Docker & Kubernetes : Horizontal pod autoscaler on minikubes

    Docker & Kubernetes : NodePort vs LoadBalancer vs Ingress

    Docker & Kubernetes : Load Testing with Locust on GCP Kubernetes

    Docker & Kubernetes : From a monolithic app to micro services on GCP Kubernetes

    Docker & Kubernetes : Rolling updates

    Docker & Kubernetes : Deployments to GKE (Rolling update, Canary and Blue-green deployments)

    Docker & Kubernetes : Slack Chat Bot with NodeJS on GCP Kubernetes

    Docker & Kubernetes : Continuous Delivery with Jenkins Multibranch Pipeline for Dev, Canary, and Production Environments on GCP Kubernetes

    Docker & Kubernetes - MongoDB with StatefulSets on GCP Kubernetes Engine

    Docker & Kubernetes : Nginx Ingress Controller on minikube

    Docker & Kubernetes : Setting up Ingress with NGINX Controller on Minikube (Mac)

    Docker & Kubernetes : Nginx Ingress Controller for Dashboard service on Minikube

    Docker & Kubernetes : Nginx Ingress Controller on GCP Kubernetes

    Docker & Kubernetes : Kubernetes Ingress with AWS ALB Ingress Controller in EKS

    Docker & Kubernetes : MongoDB / MongoExpress on Minikube

    Docker & Kubernetes : Setting up a private cluster on GCP Kubernetes

    Docker & Kubernetes : Kubernetes Namespaces (default, kube-public, kube-system) and switching namespaces (kubens)

    Docker & Kubernetes : StatefulSets on minikube

    Docker & Kubernetes : StatefulSets on minikube

    Docker & Kubernetes : RBAC

    Docker & Kubernetes Service Account, RBAC, and IAM

    Docker & Kubernetes - Kubernetes Service Account, RBAC, IAM with EKS ALB, Part 1

    Docker & Kubernetes : Helm Chart

    Docker & Kubernetes : My first Helm deploy

    Docker & Kubernetes : Readiness and Liveness Probes

    Docker & Kubernetes : Helm chart repository with Github pages

    Docker & Kubernetes : Deploying WordPress and MariaDB with Ingress to Minikube using Helm Chart

    Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 2 Chart

    Docker & Kubernetes : Deploying WordPress and MariaDB to AWS using Helm 3 Chart

    Docker & Kubernetes : Helm Chart for Node/Express and MySQL with Ingress

    Docker & Kubernetes : Docker_Helm_Chart_Node_Expess_MySQL_Ingress.php

    Docker & Kubernetes: Deploy Prometheus and Grafana using Helm and Prometheus Operator - Monitoring Kubernetes node resources out of the box

    Docker & Kubernetes : Deploy Prometheus and Grafana using kube-prometheus-stack Helm Chart

    Docker & Kubernetes : Istio (service mesh) sidecar proxy on GCP Kubernetes

    Docker & Kubernetes : Istio on EKS

    Docker & Kubernetes : Istio on Minikube with AWS EC2 for Bookinfo Application

    Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part I)

    Docker & Kubernetes : Deploying .NET Core app to Kubernetes Engine and configuring its traffic managed by Istio (Part II - Prometheus, Grafana, pin a service, split traffic, and inject faults)

    Docker & Kubernetes : Helm Package Manager with MySQL on GCP Kubernetes Engine

    Docker & Kubernetes : Deploying Memcached on Kubernetes Engine

    Docker & Kubernetes : EKS Control Plane (API server) Metrics with Prometheus

    Docker & Kubernetes : Spinnaker on EKS with Halyard

    Docker & Kubernetes : Continuous Delivery Pipelines with Spinnaker and Kubernetes Engine

    Docker & Kubernetes: Multi-node Local Kubernetes cluster - Kubeadm-dind(docker-in-docker)

    Docker & Kubernetes: Multi-node Local Kubernetes cluster - Kubeadm-kind(k8s-in-docker)

    Docker & Kubernetes : nodeSelector, nodeAffinity, taints/tolerations, pod affinity and anti-affinity - Assigning Pods to Nodes

    Docker & Kubernetes : Jenkins-X on EKS

    Docker & Kubernetes : ArgoCD App of Apps with Heml on Kubernetes

    Docker & Kubernetes : ArgoCD on Kubernetes cluster

    Docker & Kubernetes : GitOps with ArgoCD for Continuous Delivery to Kubernetes clusters (minikube) - guestbook





    Vagrant



    VirtualBox & Vagrant install on Ubuntu 14.04

    Creating a VirtualBox using Vagrant

    Provisioning

    Networking - Port Forwarding

    Vagrant Share

    Vagrant Rebuild & Teardown

    Vagrant & Ansible





    AWS (Amazon Web Services)



    AWS : EKS (Elastic Container Service for Kubernetes)

    AWS : Creating a snapshot (cloning an image)

    AWS : Attaching Amazon EBS volume to an instance

    AWS : Adding swap space to an attached volume via mkswap and swapon

    AWS : Creating an EC2 instance and attaching Amazon EBS volume to the instance using Python boto module with User data

    AWS : Creating an instance to a new region by copying an AMI

    AWS : S3 (Simple Storage Service) 1

    AWS : S3 (Simple Storage Service) 2 - Creating and Deleting a Bucket

    AWS : S3 (Simple Storage Service) 3 - Bucket Versioning

    AWS : S3 (Simple Storage Service) 4 - Uploading a large file

    AWS : S3 (Simple Storage Service) 5 - Uploading folders/files recursively

    AWS : S3 (Simple Storage Service) 6 - Bucket Policy for File/Folder View/Download

    AWS : S3 (Simple Storage Service) 7 - How to Copy or Move Objects from one region to another

    AWS : S3 (Simple Storage Service) 8 - Archiving S3 Data to Glacier

    AWS : Creating a CloudFront distribution with an Amazon S3 origin

    AWS : Creating VPC with CloudFormation

    WAF (Web Application Firewall) with preconfigured CloudFormation template and Web ACL for CloudFront distribution

    AWS : CloudWatch & Logs with Lambda Function / S3

    AWS : Lambda Serverless Computing with EC2, CloudWatch Alarm, SNS

    AWS : Lambda and SNS - cross account

    AWS : CLI (Command Line Interface)

    AWS : CLI (ECS with ALB & autoscaling)

    AWS : ECS with cloudformation and json task definition

    AWS : AWS Application Load Balancer (ALB) and ECS with Flask app

    AWS : Load Balancing with HAProxy (High Availability Proxy)

    AWS : VirtualBox on EC2

    AWS : NTP setup on EC2

    AWS: jq with AWS

    AWS : AWS & OpenSSL : Creating / Installing a Server SSL Certificate

    AWS : OpenVPN Access Server 2 Install

    AWS : VPC (Virtual Private Cloud) 1 - netmask, subnets, default gateway, and CIDR

    AWS : VPC (Virtual Private Cloud) 2 - VPC Wizard

    AWS : VPC (Virtual Private Cloud) 3 - VPC Wizard with NAT

    AWS : DevOps / Sys Admin Q & A (VI) - AWS VPC setup (public/private subnets with NAT)

    AWS : OpenVPN Protocols : PPTP, L2TP/IPsec, and OpenVPN

    AWS : Autoscaling group (ASG)

    AWS : Setting up Autoscaling Alarms and Notifications via CLI and Cloudformation

    AWS : Adding a SSH User Account on Linux Instance

    AWS : Windows Servers - Remote Desktop Connections using RDP

    AWS : Scheduled stopping and starting an instance - python & cron

    AWS : Detecting stopped instance and sending an alert email using Mandrill smtp

    AWS : Elastic Beanstalk with NodeJS

    AWS : Elastic Beanstalk Inplace/Rolling Blue/Green Deploy

    AWS : Identity and Access Management (IAM) Roles for Amazon EC2

    AWS : Identity and Access Management (IAM) Policies, sts AssumeRole, and delegate access across AWS accounts

    AWS : Identity and Access Management (IAM) sts assume role via aws cli2

    AWS : Creating IAM Roles and associating them with EC2 Instances in CloudFormation

    AWS Identity and Access Management (IAM) Roles, SSO(Single Sign On), SAML(Security Assertion Markup Language), IdP(identity provider), STS(Security Token Service), and ADFS(Active Directory Federation Services)

    AWS : Amazon Route 53

    AWS : Amazon Route 53 - DNS (Domain Name Server) setup

    AWS : Amazon Route 53 - subdomain setup and virtual host on Nginx

    AWS Amazon Route 53 : Private Hosted Zone

    AWS : SNS (Simple Notification Service) example with ELB and CloudWatch

    AWS : Lambda with AWS CloudTrail

    AWS : SQS (Simple Queue Service) with NodeJS and AWS SDK

    AWS : Redshift data warehouse

    AWS : CloudFormation - templates, change sets, and CLI

    AWS : CloudFormation Bootstrap UserData/Metadata

    AWS : CloudFormation - Creating an ASG with rolling update

    AWS : Cloudformation Cross-stack reference

    AWS : OpsWorks

    AWS : Network Load Balancer (NLB) with Autoscaling group (ASG)

    AWS CodeDeploy : Deploy an Application from GitHub

    AWS EC2 Container Service (ECS)

    AWS EC2 Container Service (ECS) II

    AWS Hello World Lambda Function

    AWS Lambda Function Q & A

    AWS Node.js Lambda Function & API Gateway

    AWS API Gateway endpoint invoking Lambda function

    AWS API Gateway invoking Lambda function with Terraform

    AWS API Gateway invoking Lambda function with Terraform - Lambda Container

    Amazon Kinesis Streams

    Kinesis Data Firehose with Lambda and ElasticSearch

    Amazon DynamoDB

    Amazon DynamoDB with Lambda and CloudWatch

    Loading DynamoDB stream to AWS Elasticsearch service with Lambda

    Amazon ML (Machine Learning)

    Simple Systems Manager (SSM)

    AWS : RDS Connecting to a DB Instance Running the SQL Server Database Engine

    AWS : RDS Importing and Exporting SQL Server Data

    AWS : RDS PostgreSQL & pgAdmin III

    AWS : RDS PostgreSQL 2 - Creating/Deleting a Table

    AWS : MySQL Replication : Master-slave

    AWS : MySQL backup & restore

    AWS RDS : Cross-Region Read Replicas for MySQL and Snapshots for PostgreSQL

    AWS : Restoring Postgres on EC2 instance from S3 backup

    AWS : Q & A

    AWS : Security

    AWS : Security groups vs. network ACLs

    AWS : Scaling-Up

    AWS : Networking

    AWS : Single Sign-on (SSO) with Okta

    AWS : JIT (Just-in-Time) with Okta



    Big Data & Hadoop Tutorials



    Hadoop 2.6 - Installing on Ubuntu 14.04 (Single-Node Cluster)

    Hadoop 2.6.5 - Installing on Ubuntu 16.04 (Single-Node Cluster)

    Hadoop - Running MapReduce Job

    Hadoop - Ecosystem

    CDH5.3 Install on four EC2 instances (1 Name node and 3 Datanodes) using Cloudera Manager 5

    CDH5 APIs

    QuickStart VMs for CDH 5.3

    QuickStart VMs for CDH 5.3 II - Testing with wordcount

    QuickStart VMs for CDH 5.3 II - Hive DB query

    Scheduled start and stop CDH services

    CDH 5.8 Install with QuickStarts Docker

    Zookeeper & Kafka Install

    Zookeeper & Kafka - single node single broker

    Zookeeper & Kafka - Single node and multiple brokers

    OLTP vs OLAP

    Apache Hadoop Tutorial I with CDH - Overview

    Apache Hadoop Tutorial II with CDH - MapReduce Word Count

    Apache Hadoop Tutorial III with CDH - MapReduce Word Count 2

    Apache Hadoop (CDH 5) Hive Introduction

    CDH5 - Hive Upgrade to 1.3 to from 1.2

    Apache Hive 2.1.0 install on Ubuntu 16.04

    Apache HBase in Pseudo-Distributed mode

    Creating HBase table with HBase shell and HUE

    Apache Hadoop : Hue 3.11 install on Ubuntu 16.04

    Creating HBase table with Java API

    HBase - Map, Persistent, Sparse, Sorted, Distributed and Multidimensional

    Flume with CDH5: a single-node Flume deployment (telnet example)

    Apache Hadoop (CDH 5) Flume with VirtualBox : syslog example via NettyAvroRpcClient

    List of Apache Hadoop hdfs commands

    Apache Hadoop : Creating Wordcount Java Project with Eclipse Part 1

    Apache Hadoop : Creating Wordcount Java Project with Eclipse Part 2

    Apache Hadoop : Creating Card Java Project with Eclipse using Cloudera VM UnoExample for CDH5 - local run

    Apache Hadoop : Creating Wordcount Maven Project with Eclipse

    Wordcount MapReduce with Oozie workflow with Hue browser - CDH 5.3 Hadoop cluster using VirtualBox and QuickStart VM

    Spark 1.2 using VirtualBox and QuickStart VM - wordcount

    Spark Programming Model : Resilient Distributed Dataset (RDD) with CDH

    Apache Spark 2.0.2 with PySpark (Spark Python API) Shell

    Apache Spark 2.0.2 tutorial with PySpark : RDD

    Apache Spark 2.0.0 tutorial with PySpark : Analyzing Neuroimaging Data with Thunder

    Apache Spark Streaming with Kafka and Cassandra

    Apache Spark 1.2 with PySpark (Spark Python API) Wordcount using CDH5

    Apache Spark 1.2 Streaming

    Apache Drill with ZooKeeper install on Ubuntu 16.04 - Embedded & Distributed

    Apache Drill - Query File System, JSON, and Parquet

    Apache Drill - HBase query

    Apache Drill - Hive query

    Apache Drill - MongoDB query









    Contact

    BogoToBogo
    contactus@bogotobogo.com

    Follow Bogotobogo

    About Us

    contactus@bogotobogo.com

    YouTubeMy YouTube channel
    Pacific Ave, San Francisco, CA 94115

    Pacific Ave, San Francisco, CA 94115

    Copyright © 2024, bogotobogo
    Design: Web Master