Fill big gaps before caulking
If the caulking keeps peeling up from the cracks in your concrete patio, try this professional solution. Use a combination of backer rod (inexpensive at home centers) and caulk. First remove the caulk that's in the gap. For latex caulk, pour caulk remover (such as Lift Off; about $6 at home centers and hardware stores; Photo 1) over the caulk, let it sit for two hours, then dig out the caulk with a stiff putty knife. For silicone or polyurethane caulk, use mineral spirits. If you're not sure what type of caulk you have, start with the caulk remover. For stubborn sections, give the remover more time to work, then continue scraping. You don't have to remove every last piece of caulk, especially if it's at the bottom of the gap, but the top 1 in. of the crack needs to be cleaned off pretty well.
Use a backer rod with a diameter that's 1/8 to 1/4 in. larger than the crack. Push the rod into the gap so the top is about 1/4 in. below the concrete surface (Photo 2). Fill the gap with caulk, them smooth the caulk with a plastic spoon. Be sure to use a caulk that's formulated for use with concrete (it'll say so on the label).