First, theories on why we even sleep:
1:- The restorative theory: Being awake and active takes a lot of energy. Aside from eating, one of the ways that our bodies conserves and restores energy and rejuvenates our body is simply by sleeping.
2:- The evolutionary theory: This is slightly linked to the restorative theory in that it revolves around survival and efficiency. When we sleep, we're not expending much energy, and we don't require much energy. Thus, by sleeping, we conserve resources to help reduce the amount of food we need to eat. Additionally, it is thought that early humans and our ancestors benefited by sleeping at night because it allowed them to rest while also remaining motionless so that predators couldn't find them.
3:- Memory consolidation theory: In short, sleep functions as a way for us to take our memories from throughout the day and sort and consolidate them so that we can remember them better. This has a rather large degree of support because some studies show that napping after studying can help increase information retention.
Onto dreams now; first, the nature of dreams. Dreams tend to be (as many I'm sure can agree with) rather emotional, not very logical, and full of sensory stimuli. These seemingly intrinsic properties can be explained with a variety of other theories:
1:- The problem solving theory: Dreams are a way that our minds take unsolved problems from throughout the day and attempt to unconsciously sort through them and look for answers. One reason this has some support is because since dreams aren't very logical, the abstract approach dreaming can lend to problem solving can sometimes provide unexplored answers by letting you think about something in a way you would've never tried otherwise.
2:- Wish fulfillment: Our dreams manifest latent desires. (Good) Dreams are a place where you can do anything, be anything, and potentially be better (in your own eyes) than the real you is. A professor once told me that "everyone is great in their dreams". Dreams can be a way for your mind to reassure itself and fulfill unlikely or impossible desires (which explains why many people fly in their dreams.
3:- Activation-synthesis theory: This is the most scientific theory that attempts to explain dreams. Essentially, it states that while you sleep and as your brain recuperates, it does whatever work it needs to do along with a little "exercising" so that your mind stays active despite your being unconscious in the form of randomly stimulating neurons. As a side effect of the random neuron firing, your cortex receives random nonsensical "messages" (for lack of a better work), and tries to make sense of the nonsense and in the process produces what we experience as dreams.
Onto the real topic of nightmares. It's a fact that people have bad dreams, but there's (are you sensing a theme here) multiple explanations for why. The strongest explanation has to do with the parts of the brain that are most active during dreams, and partially links back to some of the theories mentioned earlier. Note that all of the brain is active while we sleep, some parts are simply more or less active than others. First, recall that it is the cortex that generates the content of our dreams (that is, the cortex is what interprets the signals it's getting and turns it into something it/we can make sense of). Another part of the brain especially active while we sleep is the amygdala, which is (ding ding ding) the part of our brain most active when we are in a state of fear. This explains why nightmares are possible, because the part of our brain that responds to fear is essentially on overdrive for one reason or another. Lastly (though there is much more that can be said, I'm simply covering the most important parts of the brain in sleep), the least active portion of the brain during sleep is the frontal lobes, whose job it is to enable critical thinking - this explains why dreams are nonsensical and why we don't often realize it was a dream until we wake up because the frontal lobes aren't active and assessing the situation. All of these physiological processes combined are not only what allow dreams in general, but what give us a predisposition for bad dreams purely from the parts of the brain that contribute to dreaming in the first place. Another consideration to take is that, returning to the evolutionary theory and the problem solving theory, dreams can be considered a way for our brain to play out and determine how to react in crazy, dangerous situations without actually being in that crazy, dangerous situation, so that if it ever does occur, your brain knows how to react without thinking much. Additionally nightmares can simply be caused by stress, due to the stress temporarily wearing out the part of your brain that manages and regulates emotions, allowing your dreams (that are already emotional and nonsensical) to be a lot more spooky.
Lastly (for real this time), a brief note about why we are sometimes afraid of our thoughts, not only when looking back at a dream, but when conscious as well. All people have weird, scary thoughts sometimes. Not only about absurd dangerous hypothetical situations, our mortality, etc., but also things just like "If I did this this and this, I could rob this bank and get out totally safe and sound" for one example. It seems silly to say, but our brain essentially thinks things like this so that it has time to consider it and realize that it's what you SHOULDN'T do, and to prevent you from actually doing it. Another example is that just because sometimes you think about hitting someone that's annoying you or really want to, that doesn't mean you have anger problems, it's just your brain acknowledging something that it knows it shouldn't do but would really like to do, and making you aware of how it would play out so you realize the absurdity of the action(s) so that you DON'T do it.
Hopefully I addressed everything satisfactorily, feel free to respond with more questions that I'll do my best to answer, and if you actually read everything I said, thank you for your time. Have a nice day everyone.
TL;DR: Sleep happens, dreams happen, we have a few ideas why, no one is entirely sure, and though your brain just really likes to watch you suffer, it also is doing its best to help you survive.