Firstly, I think you had a good lesson in hazard awareness, a vital driving skill.
Cyclists are allowed to wobble, this is even written into UK law, so you must give them adequate room, ask your driving instructor, but consider giving them more room than you would passing a car.
Cyclists have every right to be upon the road, you, as the driver of over a ton of killing machine, are responsible to see that you don't injure other road users, including pedestrians.
With regard to cycle lanes, my experience is that many are not fit for the purpose, some are simply white lines painted on the road or pavement (that's side walk for our US readers), often with road furniture at inappropriate places. Often the surface is in a poor state of repair, or has the special treatment that makes riding a bike uncomfortable.
The only roads that cyclists cannot use are motorways or ones marked as no cycles.
When I was at school, we were taught to ride in a defensive manner, away from the kerb for our own protection.
I wish they would reintroduce cycle training at school.
All road users have a duty to take care.
You need to be aware of all possible hazards when driving.
I believe that people who have ridden 2 wheeled, either cycles or motorcycles make much better drivers. You should try it.
What you don't say is why the cyclist hit the kerb, was he being forced off the road by a car driver?
If there was a car in the lane next to you, you probably didn't have enough room to pass the cyclist safely, even if he hadn't fallen off and should have slowed down to drive behind him anyway.
I suggest that you get some proper driving lessons from a qualified instructor.
Get a copy of Roadcraft and the Highway code and read them both.
Don't drive again until you have read both of these and understood their contents.
You are driving a lethal weapon and are responsible for your actions.