Keith Suderman ksuderman

Parallel Tool Loading Results

Date: 2026-02-03 Cluster: ks-gcp-test (GCP) Galaxy Image: ksuderman/galaxy-guerler:26.0.dev0 Tools: ~2,776 total (~2,673 from CVMFS, 103 local)

Test Configurations

| Test | Workers | Configuration |

GCP Batch Job Runners Comparison

Galaxy supports two approaches for dispatching jobs to Google Cloud Batch: the Direct GCP Batch Runner and the Pulsar GCP Batch Runner. Each has distinct architectures and trade-offs.

Architecture Overview

Direct GCP Batch Runner (`gcp_batch`)

Runner class: galaxy.jobs.runners.gcp_batch:GoogleCloudBatchJobRunner
File access: NFS mount from Kubernetes cluster

Installations instructions

PR #314 had the unfortunate side effect of breaking some tools with permission denied or file not found errors. This affects Helms chart versions 5.10.0 through 5.14.0. Until that PR is rolled back please add the above YAML snippet when installing Galaxy to Kubernetes with the Galaxy Helm chart.

Use curl to download the nfs-fix.yml file. If you copy/paste the nfs-fix.yml file be sure to do so from the raw view

curl -o nfs-fix.yml https://gist.githubusercontent.com/ksuderman/7147b41832ab9434a243fbe58141a7a4/raw/c29820b60de205b0ab55d9d1c92acdaffa3894bb/nfs-fix.yml

Include the nfs-fix.yml file as one of the --values files in the helm install command.

The approximate number and type of each kind of tests run as GitHub actions as of Oct 30, 2023

Test	Number
API	1724
Client	834
Converter	109
DB Migration	92
Framework	360
Integration	889

job_id	tool_id	tool_version	state	memory.max_usage_in_bytes	cpuacct.usage	process_count	galaxy_slots	runtime_seconds	ref_data_size	input_data_size
ce4eadcc6a9dc62d	toolshed.g2.bx.psu.edu/repos/iuc/hisat2/hisat2/2.1.0+galaxy7	2.1.0+galaxy7	ok	10230325248.0000000	1524636164586.0000000	32.0000000	1.0000000	1467.0000000	0	1407644464
b944eb0f5f220e64	toolshed.g2.bx.psu.edu/repos/iuc/minimap2/minimap2/2.24+galaxy0	2.24+galaxy0	ok	23552430080.0000000	1284559376976.0000000	32.0000000	1.0000000	1242.0000000	0	1407644464
8510271abb9e802c	toolshed.g2.bx.psu.edu/repos/devteam/bowtie2/bowtie2/2.4.2+galaxy0	2.4.2+galaxy0	ok	8763760640.0000000	5420490778352.0000000	32.0000000	1.0000000	5462.0000000	0	1407644464
7c2d1b475743f5b6	toolshed.g2.bx.psu.edu/repos/devteam/bwa/bwa_mem/0.7.17.1	0.7.17.1	ok	12500324352.0000000	9858370786969.0000000	32.0000000	1.0000000	9451.0000000	0	1407644464

	apiVersion: v1
	kind: Pod
	metadata:
	name: gxy-galaxy-7sfwd-bcf7s
	generateName: gxy-galaxy-7sfwd-
	namespace: galaxy
	uid: 6d31be0a-146b-498a-82ad-1cd91f99d7ac
	resourceVersion: '77286'
	creationTimestamp: '2024-05-11T08:49:46Z'
	labels:

	# A simple timing test to compare lazy and greedy string interpolation
	# in log messages

	import timeit

	# Debug messages will not be logged
	SETUP = '''
	import logging
	logging.basicConfig(level=logging.WARNING)
	log = logging.getLogger("mylogger")

	#!/usr/bin/env python3

	import os
	import sys
	import yaml
	import argparse
	from jinja2 import Template
	from pprint import pprint

	def render_template(template, values):

	#!/usr/bin/env python3

	import os
	import sys
	import json
	import requests

	KB = 1024
	MB = KB * KB
	GB = MB * KB