we all have a dreamer inside of us. Although the road could be very bumpy, it does not mean it is impossible.
Since LZ knows so many people in IB. There is no need to listen to people who are not even IN. all you need to do is
1. call your ex-gf and asked them how they and their colleague get there. If they don't know anyone who has the same background as you do, that maybe a sign.... 2. get to IB's website and see what are the requirement for the jobs.
With a little effort, I am sure you will either change your mind, or stick with it and work extremely hard to get there.