Security issues

My 13-year old son raised up a question of security. I have had thoughts on that but I believe it's worth of more thoroughly thinking. What are use cases of the Black Hat visitor? What is impact of that threat?

Use case / Threat	Impact
Crashing my Home control server (later HCS)	It's not nice but doesn't cause any big harm
Getting copy of my HCS content	This is bad
Wiping content of my HCS	Not so bad, I use Amazon S3 as a backup backend
Getting copy, changing it and putting back to HCS	This is really bad. Hacker would have possibility to add his own Yubikey as authorized and get access to my house

Wow! We need to think how to make those "misuse cases" impossible.
Next we need to think about what are possible attack vectors. You must remember that Yubikey is acting as a keyboard and I'm using it in a way it's not meant to be used. What does open USB on a front door really mean?

Attack Vector ID	Attack vector	Comment
AVN	Someone get access to HCS through internet connection	Bad problem
AVM	Using mouse on a front door	It would be hard to get something useful to be done moving and clicking mouse blindly. Still I don't like the idea that someone would do that.
AVK	Using keyboard on a front door	Uups, this is bad. Alt-F4 would stop my interactive app, Ctrl-Alt-Del, Alt+T would start TaskManager etc. Really bad.
AVU	Sticking USB stick into front door socket	If my setup won't autostart anything, this is not so bad. But together with other holes this could turn into very big hole

This is a horror story what could happen if I'm not aware of security and don't find enough tricks to secure my system (maybe I will shut down this project if I get too afraid?) :

Mister Black Hat comes to my front door. He plugs a USB hub into my Yubikey socket. He sticks his USB drive and keyboard into the hub. With keyboard he hits Ctrl-ESC to get Start menu up, starts CMD.EXE and makes a wild guess: C is system, D is DVD... E is his own USB drive. He types command that copies all content to his USB drive or sends them to his internet site. Then he plugs USB drive to his own laptop, makes changes to setup, copies configuration files back to my server and opens the door with his own Yubikey ("Hello Black Hat, Welcome! Sorry, your voice box isn't configured yet but you are still welcome!").

This is a list of some of security features I have implemented (or will implement). They are listed in a table that shows Attack Vector ID form previous table and some features to work against those:

Attack Vector ID	Feature
AVN	Firewall (HW based, both IN and OUT), hardened OS, very strict control of installed software
AVM	My interactive app listening keyboard messages will set mouse position to the corner of the screen in 20 ms interval
AVK	In windows there is a system call to disable all special keys (originally meant for screen savers). I'll use that. In a tight interval I'll make sure my APP has focus. My APP does nothing else that listens keyboard messages and sends Yubikey-keystrings to my actual application service sending UDP broadcasts. I'll run it under other apps control that will restart app if is ever stopped.
AVU	Maybe I should find a registry trick, 3rd party app or some other way to disable USB drives? How about creating dummy virtual drives (SUBST) for all drive letters? ;-)

...or, to really nail it, should I setup another PC just for listening Yubikey and sending them as a UDP broadcast to Home Control server?
And finally: Did you find any other holes? Or did you find a better idea against some attack vector? Please, add a comment!