For those who are curious, here's how I'm solving this -
I'm editing the Font Awesome font files to only have the icons I want. This doesn't mess with the core and while I will have a smaller list of icons available in X, I'll also have a smaller file size.
Font Awesome files are stored on the server here: /wp-content/themes/pro/framework/fonts/font_awesome.
FontForge is a free font editor. Use it to edit the SVG files and save.
Upload your smaller SGV file to transfonter to quickly convert it to all the different font types. Upload to your server and you're done.