Why Your New External Drive is Breaking Your macOS Builds

Picture this: you're running dangerously low on disk space, so you invest in a brand new external drive. Time to migrate your development environment and get back to coding. You fire up those trusty setup scripts that have served you well for months, maybe years, and then...

Error.

What follows is the familiar developer dance of confusion and frustration. You scrutinize your scripts, double-check environment variables, recreate virtual environments, and question everything you thought you knew. In my case, I was staring at this cryptic message:

Unrecognized character \x05; marked by <-- HERE after <-- HERE near column 2 at ._Makefile.PL line 1.

The plot twist? When I recreated the exact same setup in /tmp, everything worked flawlessly.

The Mystery of the Phantom Files

A quick folder comparison revealed the smoking gun: nearly every file on my external drive had acquired a mysterious ._ companion. These weren't files I had created—they appeared seemingly out of nowhere.

The Real Culprit: macOS's Metadata Workaround for ExFAT

Here's what's actually happening behind the scenes. Most external drives ship with ExFAT formatting, which is great for cross-platform compatibility but has one significant limitation: it doesn't support Unix permissions or extended metadata that macOS relies on.

Apple's solution? Store that metadata separately in hidden ._FILENAME files. Clever, but problematic when your build tools start processing these metadata files as if they were actual source code.

When my compilation process tried to build *.Makefile.PL files, it wasn't just finding the legitimate Makefile.PL—it was also attempting to compile ._Makefile.PL, which contains binary metadata instead of valid Perl code. Hence the "unrecognized character" error.

The Simple Fix

The solution is straightforward: reformat your external drive to APFS (Apple File System). APFS natively supports the metadata and permissions that macOS expects, eliminating the need for those troublesome ._ files.

Steps to resolve:

1. Back up any important data from your external drive

2. Open Disk Utility

3. Select your external drive

4. Choose "Erase" and select APFS as the format

5. Restore your development environment

Your build scripts will thank you, and you can get back to what matters: writing code instead of debugging phantom errors caused by phantom files.

Run GUI applications inside Multipass (Ubuntu) on macOS

Multipass is a nice docker like vm platform from Ubuntu. It can configure the vm with cloud-init so you can prototype your cloud launches locally for free. 

E.g. Launch an instance (by default you get the current Ubuntu LTS)

$ multipass launch --name foo

You can read more about multipass in the docs.

Primarly it does seem like just another alternative to docker. But then I found one nice little difference, not a resource hog unlike docker and you don't run as root.

Currently to run a GUI application which you just want to try without having to clean up the system, most common approach is create a vm with VirtualBox, VMware Fusion or Parallels. But it can be a time consuming task as it installs everything from scratch.

Getting GUI applications to work on docker is possible configuration. But then, atleast on Mac, Docker is a resource hog.

With multipass I was able to get IntelliJ IDEA with following:

Prerequisites:

• Install Multipass
• Install XQuartz. Also configure XAuthLocation.

  echo "XAuthLocation /opt/X11/bin/xauth" >> ~/.ssh/config

Launch a vm with current LTS ubuntu with 4 cores CPU and 4 GB memory

$ multipass launch -n idea -c 4 -m 4G

$ multipass shell idea

$ sudo apt update

$ sudo apt install -y libxrender-dev libxtst6 libxi6 fontconfig

$ cd /tmp

$ curl -L -O https://download.jetbrains.com/idea/ideaIC-2021.1.tar.gz

$ sudo tar -xzf ideaIC-2021.1.tar.gz -C /opt

$ echo 'PATH=/opt/idea-IC-211.6693.111/bin:$PATH' >> ~/.bashrc

$ sudo apt install -y openjdk-11-jdk

You could automate the above with cloud-init. (Currently as of this writing multipass only supports yaml configs)

Now time for the magic part (on the host):

multipass shell internally is just doing a SSH

$ sudo cp '/var/root/Library/Application Support/multipassd/ssh-keys/id_rsa' ~/.ssh/multipass_id_rsa

Save this script to your PATH

$ cat ~/bin/multipass_x_ssh

 

#!/bin/bash

IP=$(multipass info $1 | grep IPv4 | awk '{ print $2 }')

ssh -X -i ~/.ssh/multipass_id_rsa ubuntu@$IP

Now when I want to run idea, all I have to do is:

$ multipass_x_ssh idea

$ idea.sh

Vmware Fusion 12 - Mac OS Big Sur - Could not connect Network

Was unable to get the network to work after a restart.

| vmx| I005: VNET: 'ethernet0' enable link state propagation, lsp.state = 5

| vmx| I005: DictionaryLoad: Cannot open file "/Library/Preferences/VMware Fusion/config": No such file or directory.

| vmx| I005: VNET: MACVNetMacosGetRealAdapterType: network type for adapter 0: 8

| vmx| I005: VNET: MACVNetMacosGetVnetProperties: vnet properties: vnet=vmnet8, nat=yes, dhcp=yes (ignored), subnet=172.16.82.0, mask=255.255.255.0, firstAddr=172.16.82.1, lastAddr=172.16.82.127, isIPv6=no, IPv6Prefix=fd15:4ba5:5a2b:1008::, IPv6PrefixLen=64

| vmx| I005: VNET: MACVNetPortVirtApiStartInterface: Waiting on semaphore for adapter: 0

| host-1216256| I005: VNET: MACVNetPortVirtApiStartHandler: starting interface for adapter: 0, status: 1009

| host-1216256| W003: VNET: MACVNetPortVirtApiStartHandler: unable to create virtual intrface for device: 0, status: 1009

| vmx| I005: VNET: Semaphore signalled for adapter: 0, timeoutMs=5000, waitMs=3

| vmx| W003: VNET: MACVNetPortVirtApiStartInterface: Failed to create interface for adapter: 0, handlerStatus: 1009

| vmx| I005: VNET: MACVNetPort_Connect: Ethernet0: can't start virtual interface

| vmx| I005: Msg_Post: Error

| vmx| I005: [msg.vnet.connectvnet] Could not connect 'Ethernet0' to virtual network '/dev/vmnet8'. More information can be found in the vmware.log file.

| vmx| I005: [msg.device.badconnect] Failed to connect virtual device 'Ethernet0'.

| vmx| I005: ----------------------------------------

Here's what worked for me:

• Click on "Apple" logo in menu bar
• Click "System Preferences"
• Click "Sharing"
• Disable "Internet Sharing"
• Restart the VM.

 

MacOS pyenv python 3.8 tkinter install

brew install pyenv

brew install tcl-tk

echo 'export PATH="$(brew --prefix tcl-tk)/bin:$PATH"' >> ~/.zshrc
export LDFLAGS="-L$(brew --prefix tcl-tk)/lib"
export CPPFLAGS="-I$(brew --prefix tcl-tk)/include"
export PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig"


PYTHON_CONFIGURE_OPTS="--with-tcltk-includes='-I$(brew --prefix tcl-tk)/include' --with-tcltk-libs='-L$(brew --prefix tcl-tk)/lib -ltcl8.6 -ltk8.6'" CFLAGS="-I$(brew --prefix tcl-tk)/include" pyenv install 3.8.3

Duply backup on MacOS

Download duplicity from http://duplicity.nongnu.org/

untar

cd duplicity-0.7.18

mkvirtualenv duply_backup

brew install librsync
brew install gpg
pip install fasteners

python setup install

Download duply from https://sourceforge.net/projects/ftplicity/files/duply%20%28simple%20duplicity%29/

untar

cd duply_2.1

workon duply_backup

./duply -V
  duply version 2.1
  (http://duply.net)

  Using installed duplicity version 0.7.18, python 2.7.14 (/Users/vkanwade/.virtualenvs/duply_backup/bin/python2), gpg 2.2.10 (Home: /Users/vkanwade/.gnupg), awk 'awk version 20070501', grep 'grep (BSD grep) 2.5.1-FreeBSD', bash '3.2.57(1)-release (x86_64-apple-darwin17)'.


export DUPL_PYTHON_BIN=~/.virtualenvs/duply_backup/bin/python
export PATH="$PATH:/opt/duply_2.1/"

duply myprofile create
vim ~/.duply/myprofile/conf

#GPG_KEY='_KEY_ID_'
GPG_PW='<some random pwd generated with pwgen. store it safe, required for restore>'

TARGET='s3://< s3_prefix>< s3_region >.amazonaws.com/< s3_bucket >/duply/myprofile'
# s3_prefix: eu-central-1 's3.', else 's3-'

export AWS_ACCESS_KEY_ID='< aws_s3_backup_id >'
export AWS_SECRET_ACCESS_KEY='< aws_s3_backup_secret >'
DUPL_PARAMS="$DUPL_PARAMS --allow-source-mismatch --s3-use-new-style --s3-european-buckets "

SOURCE='/'

MAX_AGE=1M

MAX_FULLBKP_AGE=1M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "

ARCH_DIR=~/duply-cache

NOTE: just wanted to dump the info. formatting will come.