deeTEEcee

One-time setups

1. Create a pypi account at https://pypi.org/. (If you need a separate repo for testing, you can also create it at test.pypi.org
2. Setup 2-auth and get your API key information. Save your token somewhere if needed (only accessible once)
3. Setup auth locally in ~/.pypirc

    [distutils]
    index-servers =
        pypi
 testpypi

deeTEEcee

# Sandbox k6 example for basic testing when I was migrating an on-prem client application to AWS
# `k6 run benchmark.js`

import http from 'k6/http';
import { check, sleep } from 'k6';

export let options = {
  vus: 1,
  // duration: "60s"
  iterations: 5,

deeTEEcee

Two things we can do with C and Python integration.

1. C Extensions for Python

   In other words, we can write python methods in pure C.

2. Embedding Python a

The reasons for doing #1 are obvious because we can make speed improvements. For #2, it is due to transporting Python libraries over to C.

deeTEEcee

"""
How to compare 2 csv files such that I can identify:
* Added/Modified/Removed Rows but know specifically which header changed

There are two ways to look at diffs:
1. Line-by-line diffs

Line-by-line diffs are dumb and can't tell what a "Modified" item is.

2. Diffs with primary keys.

deeTEEcee

import re
text_list = [
    "P1:field1",
    "P1:field2"
    "P2:field1"
    "P2:field2",
    "P3:field1",
    "P3:field2",
    "P4:field1",
    "P5:field1",

deeTEEcee

-- Get Postgres Summary
--
create or replace function 
count_rows(schema text, tablename text) returns integer
as
$body$
declare
  result integer;
  query varchar;
begin

deeTEEcee

# HELP go_gc_duration_seconds A summary of the GC invocation durations.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0.000244
go_gc_duration_seconds{quantile="0.25"} 0.000316201
go_gc_duration_seconds{quantile="0.5"} 0.0003472
go_gc_duration_seconds{quantile="0.75"} 0.000413601
go_gc_duration_seconds{quantile="1"} 0.010038708
go_gc_duration_seconds_sum 0.740180276
go_gc_duration_seconds_count 1456
# HELP go_goroutines Number of goroutines that currently exist.

deeTEEcee

Hex-style bit mapping

Because bytes tend to be longer, we'll want to use hex style mapping to represent bits. We know that hexadecimal is base 16 and bits are base 2 which translates to 4 bits = 1 hex.

Here's an example of why we want this case. If you have 16 bits, it looks like: 000011111111 but 0xFF is easier to read.

Here's a list of mapping for wanting only 1 bits: 0 -> 0x0 1 -> 0x1

deeTEEcee

# rs.initiate() is playing 2 roles:
# * required for my local mongo setup
# * it can also be used to test a connection but you can use almost any other evaluation command. 

# The command below runs a mongo check in a loop and times out after a minute.
# Why not use "wait-for-it.sh"? That establishes a tcp connection check but you could still run into a connection error
# if you run a single mongo connection check so we still need to loop.
timeout -v 1m bash -c 'until docker exec -it mongo mongo --eval "rs.initiate()"; do sleep 3; done'

deeTEEcee

Setting up database for testing

1. To build the image (since not hosted on dockerhub), follow the instructions on this page
   • If the dockerfile is incorrect, it's probably them just failing at README's. Try docker build -f Dockerfile.xe -t oracle/database:11.2.0.2-xe .
   • The README also provides more info. Under that page, search for the section referring to this specific image.
2. Do a test run: docker run -d -e ORACLE_PWD=oracle --shm-size="1g" -p 1521:1521 -p 8080:8080 oracle/database:11.2.0.2-xe (More details on --shm-size: oracle/docker-images#458)
   • The default database name will be 'xe.'
3. Add cx_Oracle / sqlalchemy to your system. Example here

Client installation (Debi