I'm having exactly this same problem with PB 11.1 on Win7. I've gotten a bit further, since my old XP was still available, but I haven't got it working yet either. After the APPCRASH on the installer, I found that the installation was missing these 4 files from C:\Program Files (x86)\Sybase\PowerBuilder 11.0
pb110.exe.config
PB110.lic
pb110_sysam.log
pb110_sysam.properties
I copied them over from my XP machine and started PowerBuilder. It said:
"The Sybase SySAM license was not validated. You will be given a chance to change the SySAM license."
The PowerBuilder GUI opened and it prompted me for what kind of license (standalone seat, local license). I clicked next and it prompted me to load the valid license key, which I did (I had checked in my license from my old XP and gotten a new license for my Win7 from the PowerBuilder site). Then clicked Next. It then told me "A PowerBuilder SySAM license and reference with the following characteristics will be generated. Click finish when you are ready." It was showing all the right info for my local machine. Then it said "The development environment will now shut down. The license you selected will be used when you next run." Looking good! Clicked okay. Bad news, though. The lic, config, log and properties files don't get saved (dates are unchanged from the old machine). And trying to open PowerBuilder again gets the same prompt that the license wasn't validated.
More curiousness: I tried to edit the pb.ini file to turn on logging, and it won't let me save. Something else has a handle on it. More if I figure it out...