Every boat is different and there is no best boat for all paddling conditions. Any boat is a trade off, features that work well in one set of conditions can compromise performance/handling in another set of conditions. You have to know what type of paddling conditions you want to paddle in before selecting a boat.
Multi-day expeditions dictate a different boat than morning explorations of an estuary or surf-zone excitement or teaching others to kayak.
One fundamental trade-off in boat design is tracking vs. turning. Generally a boat that tracks well (goes straight) does not turn as well as a boat that does not track well. There are varying degrees of these two characteristics in all boats, and some boats that track well can be made to turn better if you are willing and able to lean them when you turn, but if you're going to be turning a lot, buy a boat that turns, if you are going straight all day, buy a boat that tracks.
Another characteristic to consider is the initial stability of the boat. Initial (or primary) stability is the ease with which a boat starts to tip. Low initial stability will make the boat feel 'tender' or 'tippy'. A boat that is tender to sit in is going to be much more difficult to fish or take pictures out of, so if that's what you want to do, consider a boat with more initial stability. A boat with very high initial stability will be more difficult to handle in big waves, because it will tend to try to sit flat relative to the water rather than the horizon. The consequences of this tendency are left as an exercise for the reader.
Another thing to consider is the final stability of the boat. Final (or secondary) stability is the ease with which the boat tips all the way over. High final stability is desirable for any boat, but it may take some time to develop the balance and skill to take advantage of it.
Paddlers are all different. A boat will perform/handle differently for a tall person than for a short person, and for a heavy person than a light person. The fit of the cockpit will vary from boat to boat. A person's requirements for a boat may change as the person's skill level changes. Often, a person with advanced skills will be interested in different boat features than a person with beginner/intermediate skills.
********
Often people want to purchase a boat they can 'grow into'. This implies a distinction between boats that are comfortable for beginners and boats that are comfortable for experts.
The biggest perceived difference in 'beginner' boats vs. 'expert' boats is in initial stability. 'Expert' boats generally have lower initial stability than 'beginner' boats, and 'beginner' boats often increase initial stability at the expense of final stability. Advanced paddlers generally want a boat with high final stability because it is needed in more difficult sea conditions. Advanced paddlers (and beginners) also want a fast boat, and in many boats initial stability is traded off for speed.
If, as a beginner, you are willing to put up with some uneasy sensations early in your paddling career, you may wish to purchase an 'expert' boat and 'grow into' it, assuming the 'expert' boat has some other characteristics that you find desirable. Keep in mind that low initial stability, the hallmark of 'expert' boats, is not a desirable characteristic in and of itself. Find a boat that you like, and think you will continue to like as you become a better kayaker, and purchase that boat. If it happens to be a boat that is outside of your comfort level now, ask yourself honestly if it will ever be in your comfort level, and either purchase it now and put up with the difficulties, or rent/borrow boats until you are comfortable in your dream boat, then buy it.
Don't buy a boat just because someone tells you it is an 'expert' boat. Find out what you like in a boat and use your own judgement in your purchase.
Do not confuse how many years a person has been paddling with advanced skills. A person's skills will only increase if they work at increasing them.
********
Plastic is heavier, more resistant to damage, harder to repair.
Fiberglass is lighter, easier to repair, results in finer lines, but is more expensive. Fiberglass is generally more rigid than plastic, which can result in a faster boat.
Wood is labor intensive but relatively easy to build (a little less labor intensive if built from a kit), light, easy to repair, needs maintenance. There are also a few companies that manufacture wood/epoxy-construction kayaks, but they tend to be more expensive.
Fabric is labor intensive to build though a little less so than wood, fragile, and needs maintenance.
Folding boats are a form of fabric boat that collapses for transport/storage. They are generally more expensive to buy than any other kind of boat, but there are other considerations that may make them a better overall value. See section 5, folding kayaks, for more information.
Inflatable boats tend to be much less expensive than any other sort of boat.
Rigid boats may perform better than folding or inflatable boats. Folding and inflatable boats have the advantage of easier portability and storage. If you plan to travel with your boat, a folding or inflatable boat will be easier to get on airliners. If your home is tight on storage space, a folding/inflatable boat will be easier to store than a rigid boat.
More exotic materials (like kevlar, carbon fiber) tend to be lighter and costlier.
********
You can pad any boat, but it should fit you fairly well to begin with. Your contact points with the boat are your feet, your knees (on the underside of the deck), your hips (on the sides of the seat), and your butt (on the seat). Some boats fit big people better, some are better for small folks. The size of your feet is a consideration too. In general, a sea kayak needs to be comfortable because you are going to be in it all day, perhaps without a break. Some people prefer a looser fit in a sea kayak than in a whitewater boat, allowing space to stretch and move about.
Another thing to consider is cockpit size. A larger cockpit can make it easier for a person to enter and exit a boat. A smaller cockpit is preferred by some because it is considered more watertight.
********
Deck lines that run along the edges of the deck from the bow to the stern are important safety equipment. Bungies that cross the deck in front of and behind the cockpit are handy for stowing gear where it is easy to reach. Some paddlers prefer to have built-in compasses and pumps in their boats. Tow systems may be necessary for aiding other paddlers.
Different boats come with different kinds of deck rigging. Anything it doesn't have that you want you will have to add. Are you willing to go to that trouble?
********
The volume of the boat you need is dependent on how much stuff you are going to carry in it, and on how big you are (see 'fit' above). Overnight trips do not need as large a boat as week-long outings. You can, of course, pack light and get more stuff in a smaller boat (heck, Paul Caffyn has done some monstrously long trips in a Nordkapp, not the largest volume boat that's available out there), but for some people part of the joy of sea kayaking is in the amount of (luxurious) stuff they can bring. If that's you, you need a bigger boat. A bigger boat will also be easier to paddle in bigger seas than a smaller boat. Also, the way the volume of the boat is distributed is important in dictating how the boat handles, as more bow (and stern) volume helps to prevent the bow or stern of the boat from diving into the trough of waves in surf.
********
Single kayaks provide greater maneuverablity than doubles. Doubles can be faster than singles. Doubles may be able to carry more gear, but keep in mind that they need to carry more than twice as much gear for this to be true. A double will require the use of a rudder to steer. A double on a trip can provide an ill/injured person with a safer place to sit than in a single being towed. Some doubles are more stable than a single but will be more difficult to rescue and pump dry.
********
There are lots of different hatch designs out there. Considerations when looking at hatches are watertightness, resistance to breakage, and size. If you want to bring the kitchen sink, you'd better not just have a 9 inch round hatch. Consider also that heavy seas and surf can break or blow off hatch covers, so consider how they are attached to avoid losing them, and don't depend on them for floatation of the boat. If the compartments aren't full of gear, use float bags.
********
Cargo space is related to size of the boat, but also to position of the bulkheads (if there are any). The cockpit can also be used for cargo, but keep in mind that it may not stay dry, it may impede your exit if that becomes necessary, and it may fall out if you do exit. Keep in mind also that a leaky hatch or bulkhead may compromise the watertight cargo compartments, and pack accordingly.
Some sort of floatation is required for safe paddling. A "proper" sea boat should either have bulkheads that you can rely on for integrity and water-tightness, or the space forward and aft of the cockpit should be filled with secured floatation. Keep in mind that float bags take up stowage space and that stores by themselves don't fill the "holes". A sea sock is a valuable added safety measure in a boat without bow and stern bulkheads.
Almost all plastic boats have bulkheads that leak. The leaks can be repaired temporarily, but they will eventually start leaking again. Leaking bulkheads can compromise the safety provided by the added bouyancy of the watertight compartments. Expect to spend some time patching the leaks with a plastic boat.
********
This is one of sea kayaking's religious debates.
You might need a rudder to go straight, or the boat might need a rudder to go straight, or you might just want a rudder so you don't have to worry about steering. Look for a design that is durable, easily stowed, and which has a footbrace design you can live with. Like rigging, this is something you can modify if you are willing to do the work. An alternative to a rudder is a skeg, either permanent or retractable, which is basically a fixed rudder. It will not help steer, but it will help go straight. Both rudders and skegs are subject to breakage/jamming. In many rudder systems, a failure may result in losing support from your foot braces. A properly designed rudder should be able to stand up to a lot of abuse including resting the kayak on end on it.
A rudder should not be necessary for you to control your kayak, and you should learn proper kayaking technique without the rudder becoming a crutch.
Two boat characteristics that a rudder or skeg can help with are the boat's tendency to weathercock, and the boat's tendency to broach.
Weathercocking occurs when there is a wind in the front quarter or beam of the boat. Because of their aerodynamics/hydrodynamics, many boats will tend to try to turn into a wind when they are moving forward because the bow of the boat is held in place by the bow wave generated by the boat's forward movement, while the stern is free to pivot. A boat that weathercocks is safer than one in which the bow is blown downwind as it is very difficult to turn a boat with this characteristic into the wind.
Broaching is the boat's tendency to turn sideways to a wave coming from the stern or rear quarter of the boat. This happens because the water in waves is moving more slowly in the trough of the wave than at the crest, making the stern of the boat try to 'catch up' to the bow.
********
If you need to haul the boat on and off the top of the car, carry it any distance, or portage, this is an important characteristic. Lighter boats also tend to feel livelier in the water and are faster, although this is not as much of a consideration when you've got 300 pounds of boater and gear in the boat. In general, plastic is heavier than fiberglass is heavier than exotic materials like kevlar, carbon fiber, etc., but there are exceptions.
********
If you want to drag your boat over rocks or drop it off a pier, this is an important consideration, but even if you don't abuse your boat, it wears in normal usage as well, so consider durability in your selection. In general, plastic stands up to abuse better than fiberglass, but is harder to repair. Keep in mind that in fiberglass construction, heavier is not necessarily stronger.
********
Buy a boat you can afford, but if you find a boat you really like which is too expensive, it may be worthwhile to save your pennies until you can afford it. If you have a fixed price range you are interested in, it may be a good idea to only try boats in that price range, so you are satisfied with what you get. Plastic boats run $700-$1500, Fiberglass $1300-$2800, other materials tend to cost more. Sometimes you can find boats sold used for less, especially if a shop or outfitter is selling old boats from their rental fleet.
********
The best way to choose a boat, and the only way to determine its paddling characteristics, is to try it, and you should take opportunities to try as many boats as you can to decide what you like. Many shops have demo days, and symposia are good opportunities to try boats. Try to find an opportunity to paddle in the conditions you are planning on using the boat in. Also, consider how the boat handles when it is loaded as well as unloaded. Things to think about when you are trying the boat are:
Does it feel comfortable just sitting in it?
Lean the boat onto it's side. Does it stop leaning or keep going and tip over? Is there a point where the resistance to leaning increases?
Paddle the boat into the wind, across the wind, with the wind behind you. How easy is it to keep on course? Does it turn into the wind (weathercocking) or out of the wind excessively? How fast is it?
How easy is the boat to turn?