[NOTE: This information was updated in October 2014.]
I live in the state of Washington. I live in Washington State. And when it comes to capitalization, it's the state of confusion. Why? Because various style manuals recommend capitalizing differently.
When state comes after the state's name, it is capitalized, according to The Gregg Reference Manual and The Chicago Manual of Style:
New York State
That's the style I follow. However, you may wish to follow the style of the Associated Press Stylebook (AP), which does not capitalize state after the name:
New York state
AP points out that "Washington State" is the name of a university.
We who live in Washington State and New York State come up against this question often. We are the only states in the U.S. that need to distinguish ourselves from other geographical entities: Washington, D.C., and New York City. Colorado State, Michigan State, and other such names do not refer to a state, but rather a college or university. (Yes, the word State is still capitalized.)
When state comes before the state's name, all three style manuals agree that it is lower case, that is, not capitalized:
state of Washington
state of New York
the great state of Texas
the sunny state of California
For those beyond the complexity of Washington State (state?) and New York State (state?), there are these rules too:
State is generally not capitalized when it is used in place of the state name, as in "This is a new state program" or "He is a state employee."
States is capitalized when it refers to the United States.
Of course, all key words are capitalized in a heading, for example, State of Florida Employment Application.
The Chicago Manual of Style offers more details on the topic. It states, "Where a specific governmental body rather than the place is meant, the words state, city, and the like are usually capitalized when used as part of the full name of the body." Chicago gives these examples:
She works for the Village of Forest Park.
That is a City of Chicago ordinance.
But: Residents of the village of Forest Park enjoy easy access to the city of Chicago.
Applying the Chicago rule, do I live in the City of Seattle? No, it's simply the city of Seattle. Capitalize city only when it is part of the city's name (New York City, Kansas City, Sun City), its creative name (Windy City for Chicago), or the name of the governmental body.
"AP" provides these examples of governmental bodies:
Boston City Council
New York State Police
state Transportation Department (where "state" is not part of the name)
I hope you are now in a state of clarity.