Is It Possible to Add Too Much Cayenne Pepper to Soup?

Yes.  It is. 

I made "Bretzenheim soup" today.  This is something I made when I was a postdoc in Germany -- Bretzenheim is a suburb of Mainz.  The soup is just dried or frozen tortellini (I used cheese ravioli because I could not find the tortellini at the supermarket I was in yesterday) boiled until done in about 2 quarts of water with one package of onion soup mix.  A couple of minutes before it's done, add a few cups of frozen peas.  (Note that I'm not really adhering to a detailed recipe or detailed measurements.  This is bachelor cooking, for crying out loud!)

The last important touch is a dash of cayenne pepper.  Originally, I had planned to add paprika, but I grabbed the wrong spice by mistake, and I liked it having a bit more of a kick.  

That said, there is such a thing as too much of a good thing.