Kyle went to Stars Hollow High School and was good friends with Dean. He also hosted the party behind his parents back. At this party, Dean and Jess get in a physical fight after Jess yells at Rory, causing her to cry. Dean and Jess cause a lot of damage to Kyle's house, including breaking a fence. The police break up the party. This party was the main reason Jess left town, causing Rory and him to break up.
Kyle's parents found out about the party and Kyle entered the Navy (unclear if this was because of the party itself). Kyle was part of Dean's bachelor party (when Dean drunkenly confesses his love for Rory to Luke). Dean also moved in with Kyle after Lindsay and him separated. While Dean lives with Kyle, we found out Kyle was wounded in the Navy and now has a hook for one of his hands. He finds that the hook helps him to meet many women.
The last time that Kyle is seen is near the end of Season 5 when he is volunteering for Taylor along with Luke, Dean, Gypsy, and Kirk to turn the late Mr Twickam's house into a town museum. Kyle and Kirk annoy Taylor by inappropriately playing with the mannequins and there is an awkward moment when they all hold hands in a circle and Luke has to hold Kyle's hook.