A functional programming language is a language which focuses on the idea of using and composing "functions" as the fundamental tool of abstraction. A function is a piece of code with parameters and produces a result based solely on those parameters. A function is also a first-class value in these languages and may be provided as the argument to or result of another function.
Now there is a broad spectrum of functional programming languages from there. Some adhere strictly to the notion of a function, and others are less restrictive. In addition, many functional programming languages are designed with advanced static type and module systems, but not all of them are. There is a clever correspondence connecting functional programs using parametric polymorphic types to proofs and propositions in intuitionistic logic. If you are interested in writing computer-verified logical proofs, then this is an area to explore further.
Of the dynamically typed functional programming languages, the recommended starting point is Scheme, as mentioned above. There are many excellent resources, such as this book online:
http://www.htdp.org. How to Design Programs. You should always use a Lisp-aware editor for writing Lisp-like languages such as Scheme (emacs, vim, etc... have very good support).
Of the statically typed functional programming languages, most people start with either ML or Haskell. OCaml is the most widely used ML, and Standard ML is mostly used for teaching these days, but is very powerful. Haskell is a very interesting language because it is considered "pure" FP and tries to take advantage of that. This includes using so-called "lazy evaluation" instead of the typical call-by-value, and also there are many advanced type system experiments being conducted in the real-world high-performance compiler, GHC. The Haskell community is probably the largest of the FP languages as well, there is a lot of support. Start at
http://www.haskell.org/.