A power plant supplying base load power needs to be able to run for months on end without needing to be taken down for maintenance, and it's best if the fuel costs are relatively low. However, since base load power plants are rarely taken offline, it's not a huge problem if it takes them a while to start up.
A peaking power plant is one we can switch on when we need additional power, which will come online without much delay and start generating power on a moments' notice. As peaker plants are used for less time over the course of a year, it's not as crucial that the cost of fuel be low.
Typical base load power plants are coal-fired, nuclear and hydroelectric. Geothermal can also provide base load power. Base load power plants tend to be expensive to build, and coal and nuclear take days to reach full power once fired up. But fuel costs per kilowatt generated tend to be low, at least if you don't count the ecological costs.
Peaking power plants have traditionally been fueled by either natural gas, diesel oil, or jet fuel. The last two are significantly more expensive than gas, especially since the advent of fracking has pushed natural gas prices through the floor. Most peak power in the US comes from gas-fired plants. Despite gas' low price these days, peak power remains more expensive per kilowatt than base load. Hydro can also be used as a peak power source, as ramping up power production from a hydroelectric dam is generally a matter of letting a bit more water in
Solar power plants, by virtue of using the sun as fuel directly, can only produce power when the sun is shining (or, if expensive storage is added to a solar thermal plant, for a few hours afterward). As sunny afternoon hours more or less coincide with peak electrical demand, solar power plants are peaker plants, and will be until engineers make either thermal or grid storage a reality.
There's also an intermediate kind of power plant, referred to as "load-following" plants, in areas with high electrical demand. Load-following plants supplement the power produced by base load plants, but run for longer periods of time during a typical day -- or 24 hours, but with lower output at night. In the US, load-following plants are generally gas-fired or hydro, though nuclear-heavy Chicago does use nukes as load-following plants as well.
Wind, by the way, is an odd person out. In most places it's not constant enough to be base load, and not reliable enough to provide a secure source of peaking power. If we get to the point where large numbers of wind turbines in widely separated locations are all hooked together through the grid, some of that unpredictability will go away: wind dying down in one place might well be made up for when it picks up somewhere else. For now, wind tends to be a continually moving monkey wrench in the engineers' careful daily planning of power supply and demand.
