Zengtudor/windows_in_docker
1
0
Fork 0
mirror of https://github.com/dockur/windows.git synced 2026-02-03 17:27:21 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix: Cache original ISOs instead of rebuilt ones, add cache download

Browse Source 
The container's genisoimage rebuilds ISOs with a duplicate boot catalog
entry that 7z cannot re-extract ("Break signaled"). This caused cache
restore to fail for new instances.

- Add `cache download <version>` command to download original ISOs
  directly using the container's download logic
- Add `_is_rebuilt_iso()` helper to detect rebuilt ISOs (magic byte 0x16)
- Skip rebuilt ISOs in `cache save` and `auto_cache_save` with warning
- Remove magic byte manipulation from restore paths (originals work as-is)
- Update WINCTL_GUIDE.md and readme.md with new cache workflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Michel Abboud 2026-01-30 12:41:40 +00:00
parent 04d909acd7
commit 2e51fafe80
3 changed files with 205 additions and 72 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
WINCTL_GUIDE.md
View File

@ -702,21 +702,29 @@ Windows ISOs are large (3-6 GB) and re-downloaded every time a new container is
### How It Works ### How It Works
1. Start a VM and wait for the ISO to download 1. Download the original ISO: `./winctl.sh cache download winxp`
2. Cache the ISO: `./winctl.sh cache save winxp` 2. Create new instances — cached ISOs are auto-restored: `./winctl.sh start winxp --new`
3. Create new instances — cached ISOs are auto-restored: `./winctl.sh start winxp --new`
When creating a new instance with `--new`, winctl checks `cache/<version>/` for ISOs and metadata files, then copies them into the new instance's data directory before starting the container. The container recognizes the ISO as already processed and boots directly — no download or re-extraction needed. When creating a new instance with `--new`, winctl checks `cache/<version>/` for ISOs and copies them into the new instance's data directory before starting the container. The container finds the ISO locally and processes it (extracts, injects drivers, builds answer file) without needing to re-download.
### Caching an ISO ### Downloading an ISO
```bash ```bash
# Cache ISOs from an existing VM's data directory # Download original ISOs to the cache
./winctl.sh cache save winxp ./winctl.sh cache download winxp
./winctl.sh cache save win11 ./winctl.sh cache download win11
``` ```
The ISOs and metadata files (`windows.base`, `windows.ver`, `windows.mode`, `windows.type`, `windows.args`) are copied from `data/<name>/` to `cache/<base-version>/`. This uses the container's download logic to fetch the original, unprocessed ISO directly to the cache. This is the recommended way to populate the cache.
### Saving from an Existing VM
```bash
# Cache unprocessed ISOs from a VM's data directory
./winctl.sh cache save winxp
```
> **Note:** After a VM completes its first boot, the container rebuilds the ISO with injected drivers. These rebuilt ISOs are automatically skipped by `cache save` because they cannot be re-processed. Use `cache download` instead.
### Listing Cached ISOs ### Listing Cached ISOs
@ -743,11 +751,11 @@ Both commands require typing `yes` to confirm.
When creating a new instance with `--new` (without `--clone`), winctl automatically checks the cache: When creating a new instance with `--new` (without `--clone`), winctl automatically checks the cache:
```bash ```bash
# If cache/winxp/ has ISOs + metadata, they are copied to data/winxp-1/ before start # If cache/winxp/ has an ISO, it is copied to data/winxp-1/ before start
./winctl.sh start winxp --new ./winctl.sh start winxp --new
``` ```
Both the ISO and metadata files are restored so the container recognizes the ISO as already processed. This is skipped when using `--clone`, since cloning copies all data from the base version including any ISOs. The original ISO is copied as-is. The container processes it locally (extract, inject drivers, build answer file) without needing to re-download. This is skipped when using `--clone`, since cloning copies all data from the base version including any ISOs.
### Auto-Cache on Stop ### Auto-Cache on Stop
@ -758,28 +766,21 @@ To automatically cache ISOs whenever a container is stopped, add `AUTO_CACHE=Y`
AUTO_CACHE=Y AUTO_CACHE=Y
``` ```
When enabled, `winctl.sh stop` will silently cache any ISOs found in the stopped container's data directory. ISOs that are already cached are skipped. This removes the need to manually run `cache save` after the first download. When enabled, `winctl.sh stop` will silently cache any unprocessed ISOs found in the stopped container's data directory. Rebuilt ISOs (from the container's install pipeline) and already-cached ISOs are skipped.
### Cache Directory Structure ### Cache Directory Structure
``` ```
cache/ cache/
├── winxp/ ├── winxp/
│ ├── winxpx86.iso │ └── winxpx86.iso
│ ├── windows.base
│ ├── windows.ver
│ ├── windows.mode
│ ├── windows.type
│ └── windows.args
├── win11/ ├── win11/
│ ├── win11x64.iso │ └── win11x64.iso
│ └── ...
└── win10/ └── win10/
├── win10x64.iso └── win10x64.iso
└── ...
``` ```
> **Note:** The cache stores processed ISOs and their metadata from the container's data directory, not raw downloads. Metadata files tell the container the ISO is already processed, so it boots directly without re-extracting or re-downloading. > **Note:** The cache must contain original (unprocessed) ISOs. Use `cache download` to populate it. Rebuilt ISOs from `data/` directories are skipped by `cache save` because the container cannot re-extract them.
--- ---

4
readme.md
View File

@ -130,11 +130,11 @@ Use `winctl.sh` for easy container management:
./winctl.sh destroy winxp-lab # Remove instance ./winctl.sh destroy winxp-lab # Remove instance
# ISO cache (skip re-downloads for new instances) # ISO cache (skip re-downloads for new instances)
./winctl.sh cache save winxp # Cache ISO after first download ./winctl.sh cache download winxp # Download original ISO to cache
./winctl.sh cache list # Show cached ISOs ./winctl.sh cache list # Show cached ISOs
./winctl.sh start winxp --new # New instance uses cached ISO
./winctl.sh cache rm winxp # Remove cached winxp ISO ./winctl.sh cache rm winxp # Remove cached winxp ISO
./winctl.sh cache flush # Clear all cached ISOs ./winctl.sh cache flush # Clear all cached ISOs
# Or set AUTO_CACHE=Y in .env to cache ISOs automatically on stop
# Full help (includes ARM64 info) # Full help (includes ARM64 info)
./winctl.sh help ./winctl.sh help

226
winctl.sh
View File

@ -1092,17 +1092,10 @@ cmd_start() {
local cached_iso local cached_iso
cached_iso=$(find "$cache_src" -maxdepth 1 -name '*.iso' -type f -print -quit 2>/dev/null || true) cached_iso=$(find "$cache_src" -maxdepth 1 -name '*.iso' -type f -print -quit 2>/dev/null || true)
if [[ -n "$cached_iso" ]]; then if [[ -n "$cached_iso" ]]; then
# Keep original filename — the container's parseVersion
# determines the expected name (e.g. winxpx86.iso, not xp.iso)
local iso_name local iso_name
iso_name=$(basename "$cached_iso") iso_name=$(basename "$cached_iso")
info "Restoring cached ISO: $iso_name..." info "Restoring cached ISO: $iso_name..."
cp "$cached_iso" "$data_dir/$iso_name" cp "$cached_iso" "$data_dir/$iso_name"
# Restore metadata so the container recognizes the ISO as processed
local _meta
for _meta in windows.base windows.ver windows.mode windows.type windows.args; do
[[ -f "$cache_src/$_meta" ]] && cp "$cache_src/$_meta" "$data_dir/$_meta"
done
success "ISO restored from cache (skipping download)" success "ISO restored from cache (skipping download)"
fi fi
fi fi
@ -2397,17 +2390,10 @@ cmd_new() {
local cached_iso local cached_iso
cached_iso=$(find "$cache_src" -maxdepth 1 -name '*.iso' -type f -print -quit 2>/dev/null || true) cached_iso=$(find "$cache_src" -maxdepth 1 -name '*.iso' -type f -print -quit 2>/dev/null || true)
if [[ -n "$cached_iso" ]]; then if [[ -n "$cached_iso" ]]; then
# Keep original filename — the container's parseVersion
# determines the expected name (e.g. winxpx86.iso, not xp.iso)
local iso_name local iso_name
iso_name=$(basename "$cached_iso") iso_name=$(basename "$cached_iso")
info "Restoring cached ISO: $iso_name..." info "Restoring cached ISO: $iso_name..."
cp "$cached_iso" "$data_dir/$iso_name" cp "$cached_iso" "$data_dir/$iso_name"
# Restore metadata so the container recognizes the ISO as processed
local _meta
for _meta in windows.base windows.ver windows.mode windows.type windows.args; do
[[ -f "$cache_src/$_meta" ]] && cp "$cache_src/$_meta" "$data_dir/$_meta"
done
success "ISO restored from cache (skipping download)" success "ISO restored from cache (skipping download)"
fi fi
fi fi
@ -2597,7 +2583,17 @@ cmd_instances() {
# ISO CACHE # ISO CACHE
# ============================================================================== # ==============================================================================
# Non-interactive cache save — silently skips if no ISOs or already cached. # Check if an ISO has been rebuilt by the container (magic byte 0x16 at offset 0).
# Rebuilt ISOs cannot be re-processed by the container's install pipeline (7z
# fails on the duplicate boot catalog entry created by genisoimage).
_is_rebuilt_iso() {
local iso="$1"
local magic
magic=$(dd if="$iso" bs=1 count=1 status=none 2>/dev/null | od -A n -t x1 -v | tr -d ' \n')
[[ "$magic" == "16" ]]
}
# Non-interactive cache save — silently skips rebuilt ISOs and already-cached files.
# Used by cmd_stop() when AUTO_CACHE=Y. # Used by cmd_stop() when AUTO_CACHE=Y.
auto_cache_save() { auto_cache_save() {
local target="$1" local target="$1"
@ -2614,19 +2610,13 @@ auto_cache_save() {
while IFS= read -r iso; do while IFS= read -r iso; do
local filename local filename
filename=$(basename "$iso") filename=$(basename "$iso")
# Skip rebuilt ISOs — they can't be re-processed by the container
_is_rebuilt_iso "$iso" && continue
if [[ ! -f "$cache_dest/$filename" ]]; then if [[ ! -f "$cache_dest/$filename" ]]; then
cp "$iso" "$cache_dest/$filename" cp "$iso" "$cache_dest/$filename"
info "Auto-cached ISO: $filename" info "Auto-cached ISO: $filename"
fi fi
done <<< "$iso_files" done <<< "$iso_files"
# Cache metadata files so the container recognizes the ISO as processed
local meta
for meta in windows.base windows.ver windows.mode windows.type windows.args; do
if [[ -f "$data_dir/$meta" ]] && [[ ! -f "$cache_dest/$meta" ]]; then
cp "$data_dir/$meta" "$cache_dest/$meta"
fi
done
} }
cmd_cache() { cmd_cache() {
@ -2634,18 +2624,19 @@ cmd_cache() {
shift || true shift || true
case "$subcmd" in case "$subcmd" in
save) cmd_cache_save "$@" ;; save) cmd_cache_save "$@" ;;
list) cmd_cache_list ;; download) cmd_cache_download "$@" ;;
rm) cmd_cache_rm "$@" ;; list) cmd_cache_list ;;
flush) cmd_cache_flush ;; rm) cmd_cache_rm "$@" ;;
flush) cmd_cache_flush ;;
"") "")
error "Missing subcommand" error "Missing subcommand"
printf '%s\n' " Usage: ${SCRIPT_NAME} cache <save|list|rm|flush>" printf '%s\n' " Usage: ${SCRIPT_NAME} cache <save|download|list|rm|flush>"
exit 1 exit 1
;; ;;
*) *)
error "Unknown cache subcommand: $subcmd" error "Unknown cache subcommand: $subcmd"
printf '%s\n' " Usage: ${SCRIPT_NAME} cache <save|list|rm|flush>" printf '%s\n' " Usage: ${SCRIPT_NAME} cache <save|download|list|rm|flush>"
exit 1 exit 1
;; ;;
esac esac
@ -2681,9 +2672,15 @@ cmd_cache_save() {
header "Cache Save: ${RESOLVED_DISPLAY_NAME}" header "Cache Save: ${RESOLVED_DISPLAY_NAME}"
local count=0 local count=0
local rebuilt=false
while IFS= read -r iso; do while IFS= read -r iso; do
local filename local filename
filename=$(basename "$iso") filename=$(basename "$iso")
if _is_rebuilt_iso "$iso"; then
warn "Skipping $filename — rebuilt ISO (cannot be re-processed)"
rebuilt=true
continue
fi
if [[ -f "$cache_dest/$filename" ]]; then if [[ -f "$cache_dest/$filename" ]]; then
info "Already cached: $filename" info "Already cached: $filename"
else else
@ -2697,16 +2694,144 @@ cmd_cache_save() {
((count++)) ((count++))
done <<< "$iso_files" done <<< "$iso_files"
# Cache metadata files so the container recognizes the ISO as processed printf '\n'
local meta if (( count > 0 )); then
for meta in windows.base windows.ver windows.mode windows.type windows.args; do success "Cached $count ISO file(s) to cache/$RESOLVED_BASE/"
if [[ -f "$data_dir/$meta" ]] && [[ ! -f "$cache_dest/$meta" ]]; then fi
cp "$data_dir/$meta" "$cache_dest/$meta" if [[ "$rebuilt" == true ]]; then
fi warn "Some ISOs were skipped because they were rebuilt by the container."
done info "Use '${SCRIPT_NAME} cache download $RESOLVED_BASE' to download the original ISO."
fi
printf '\n'
}
cmd_cache_download() {
local target="${1:-}"
if [[ -z "$target" ]]; then
error "Missing version"
printf '%s\n' " Usage: ${SCRIPT_NAME} cache download <version>"
exit 1
fi
# Resolve the base version name
local base="$target"
if [[ -v VERSION_ENV_VALUES[$target] ]]; then
base="$target"
else
# Try to find a matching version key
local found=""
for k in "${!VERSION_DISPLAY_NAMES[@]}"; do
if [[ "$k" == "$target" ]]; then
found="$k"
break
fi
done
[[ -n "$found" ]] && base="$found"
fi
# Use the container to resolve the download URL and filename
local windows_image
local resource_type="${VERSION_RESOURCE_TYPE[$base]:-modern}"
if [[ "$resource_type" == "legacy" ]]; then
windows_image=$(grep -E '^WINDOWS_IMAGE=' "$SCRIPT_DIR/.env.legacy" 2>/dev/null | tail -1 | cut -d'=' -f2- || true)
else
windows_image=$(grep -E '^WINDOWS_IMAGE=' "$SCRIPT_DIR/.env.modern" 2>/dev/null | tail -1 | cut -d'=' -f2- || true)
fi
windows_image="${windows_image:-dockurr/windows}"
local version_env="${VERSION_ENV_VALUES[$base]:-$base}"
header "Cache Download: ${VERSION_DISPLAY_NAMES[$base]:-$base}"
info "Resolving download URL..."
local iso_filename
iso_filename=$(docker run --rm --entrypoint="" -e "VERSION=$version_env" "$windows_image" bash -c '
set +eu
APP="Windows"
source /run/utils.sh 2>/dev/null
source /run/define.sh 2>/dev/null
parseVersion 2>/dev/null
echo "${VERSION//\//}.iso"
' 2>/dev/null || true)
if [[ -z "$iso_filename" ]]; then
error "Could not resolve ISO filename for $base"
exit 1
fi
local cache_dest="$ISO_CACHE_DIR/$base"
mkdir -p "$cache_dest"
if [[ -f "$cache_dest/$iso_filename" ]] && ! _is_rebuilt_iso "$cache_dest/$iso_filename"; then
info "Already cached: $iso_filename"
local size
size=$(du -h "$cache_dest/$iso_filename" | awk '{print $1}')
printf '%s\n' " Size: $size"
printf '\n'
return 0
fi
# Run the container briefly to download the ISO, then copy from its tmp dir
local tmp_dir
tmp_dir=$(mktemp -d)
info "Downloading original ISO (this may take a while)..."
info "Using container to download ${VERSION_DISPLAY_NAMES[$base]:-$base}..."
# Run container with storage mounted, let it download, then grab from tmp
# The container downloads to /storage/tmp/<filename>.iso, so we watch for it
local container_name="winctl-download-$$"
docker run -d --rm --entrypoint="" \
--name "$container_name" \
-e "VERSION=$version_env" \
-v "$tmp_dir:/storage" \
"$windows_image" bash -c '
set +eu
APP="Windows"
STORAGE="/storage"
source /run/utils.sh 2>/dev/null
source /run/define.sh 2>/dev/null
source /run/mido.sh 2>/dev/null
source /run/install.sh 2>/dev/null
parseVersion 2>/dev/null
BOOT="$STORAGE/${VERSION//\//}.iso"
TMP="$STORAGE/tmp"
mkdir -p "$TMP"
ISO=$(basename "$BOOT")
ISO="$TMP/$ISO"
if [ -f "$BOOT" ] && [ -s "$BOOT" ]; then
mv -f "$BOOT" "$ISO"
fi
if [ ! -s "$ISO" ] || [ ! -f "$ISO" ]; then
downloadImage "$ISO" "$VERSION" "en" || exit 1
fi
# Signal completion — move ISO out of tmp to storage root
cp "$ISO" "$BOOT"
' > /dev/null 2>&1
# Wait for the download container to finish
docker wait "$container_name" > /dev/null 2>&1
# Check if the ISO was downloaded
local downloaded_iso
downloaded_iso=$(find "$tmp_dir" -maxdepth 1 -name '*.iso' -type f -print -quit 2>/dev/null || true)
if [[ -z "$downloaded_iso" ]] || [[ ! -s "$downloaded_iso" ]]; then
rm -rf "$tmp_dir"
error "Failed to download ISO for $base"
exit 1
fi
local dl_name
dl_name=$(basename "$downloaded_iso")
cp "$downloaded_iso" "$cache_dest/$dl_name"
rm -rf "$tmp_dir"
local size
size=$(du -h "$cache_dest/$dl_name" | awk '{print $1}')
printf '\n' printf '\n'
success "Cached $count ISO file(s) to cache/$RESOLVED_BASE/" success "Downloaded and cached: $dl_name ($size)"
printf '\n' printf '\n'
} }
@ -2905,7 +3030,7 @@ _help_summary() {
_help_row "clean [--data]" "Remove stopped containers" _help_row "clean [--data]" "Remove stopped containers"
_help_row "destroy <instance>" "Permanently remove an instance" _help_row "destroy <instance>" "Permanently remove an instance"
_help_row "instances [base]" "List all registered instances" _help_row "instances [base]" "List all registered instances"
_help_row "cache <sub>" "Manage ISO cache (save/list/rm/flush)" _help_row "cache <sub>" "Manage ISO cache (download/save/list/rm/flush)"
_help_row "help [topic]" "Show help (topics below, or 'all')" _help_row "help [topic]" "Show help (topics below, or 'all')"
printf '\n' printf '\n'
printf '%b\n' "${BOLD}QUICK START${RESET}" printf '%b\n' "${BOLD}QUICK START${RESET}"
@ -2985,7 +3110,7 @@ _help_topic_commands() {
_help_row "clean [--data]" "Remove stopped containers" _help_row "clean [--data]" "Remove stopped containers"
_help_row "destroy <instance>" "Permanently remove an instance" _help_row "destroy <instance>" "Permanently remove an instance"
_help_row "instances [base]" "List all registered instances" _help_row "instances [base]" "List all registered instances"
_help_row "cache <sub>" "Manage ISO cache (save/list/rm/flush)" _help_row "cache <sub>" "Manage ISO cache (download/save/list/rm/flush)"
_help_row "help [topic]" "Show help (topics: commands, instances, cache, examples, config, all)" _help_row "help [topic]" "Show help (topics: commands, instances, cache, examples, config, all)"
printf '\n' printf '\n'
printf '%b\n' "${BOLD}CATEGORIES${RESET}" printf '%b\n' "${BOLD}CATEGORIES${RESET}"
@ -3027,22 +3152,29 @@ _help_topic_instances() {
_help_topic_cache() { _help_topic_cache() {
printf '%b\n' "${BOLD}CACHE COMMANDS${RESET}" printf '%b\n' "${BOLD}CACHE COMMANDS${RESET}"
printf '\n' printf '\n'
_help_row "cache save <version>" "Cache ISOs from a VM data directory" _help_row "cache download <version>" "Download original ISO to cache"
_help_row "cache list" "Show all cached ISOs with sizes" _help_row "cache save <version>" "Cache ISOs from a VM data directory"
_help_row "cache rm <version>" "Remove cached ISOs for a version" _help_row "cache list" "Show all cached ISOs with sizes"
_help_row "cache flush" "Clear all cached ISOs" _help_row "cache rm <version>" "Remove cached ISOs for a version"
_help_row "cache flush" "Clear all cached ISOs"
printf '\n' printf '\n'
printf '%b\n' "${BOLD}CACHE EXAMPLES${RESET}" printf '%b\n' "${BOLD}CACHE EXAMPLES${RESET}"
printf '\n' printf '\n'
printf ' %s cache save winxp # Cache ISO after first download\n' "${SCRIPT_NAME}" printf ' %s cache download winxp # Download original XP ISO to cache\n' "${SCRIPT_NAME}"
printf ' %s cache list # Show cached ISOs\n' "${SCRIPT_NAME}" printf ' %s cache list # Show cached ISOs\n' "${SCRIPT_NAME}"
printf ' %s start winxp --new # New instance uses cached ISO\n' "${SCRIPT_NAME}"
printf ' %s cache rm winxp # Remove cached winxp ISO\n' "${SCRIPT_NAME}" printf ' %s cache rm winxp # Remove cached winxp ISO\n' "${SCRIPT_NAME}"
printf ' %s cache flush # Clear all cached ISOs\n' "${SCRIPT_NAME}" printf ' %s cache flush # Clear all cached ISOs\n' "${SCRIPT_NAME}"
printf '\n' printf '\n'
printf '%b\n' "${BOLD}AUTO-CACHE${RESET}" printf '%b\n' "${BOLD}HOW IT WORKS${RESET}"
printf '\n' printf '\n'
printf ' Set AUTO_CACHE=Y in .env to automatically cache ISOs on stop.\n' printf ' The cache stores original (unprocessed) ISOs. When creating a new\n'
printf ' Cached ISOs are auto-restored when creating new instances with --new.\n' printf ' instance, the cached ISO is copied to the data directory. The container\n'
printf ' processes it locally (extract, inject drivers, answer file) without\n'
printf ' needing to re-download.\n'
printf '\n'
printf ' Use "cache download" to pre-download ISOs. "cache save" only works\n'
printf ' if the data directory has an unprocessed ISO (skips rebuilt ones).\n'
printf '\n' printf '\n'
} }