Treo online international
One company in Australia offers a year's access for $A30 including occasional calls and occasional data downloads, such as webmail retrieval or sending. This much won't last long if used often, but suits occasional use. Similar deals can be found in other countries.
When travelling internationally, buy a new prepaid card for each country as global roaming charges are far too high.
If travelling within one country, a post paid plan can offer what you need if accessing internet and email regularly. Beware that providers only provide access to their own email accounts so you may have to use internet mail if you have separate providers (see update below for correction). Retrieving your bigpond email from a vodafone server usually will not work.
Accessing your email is in fact possible from an internet connection using any server. It is necessary to set up your email client for authentification of out-going mail. This is usually done by checking a box which says something like "server requires authentification" in the out-going mail server settings. ISPs may not require this for the email of users of their own network, but when accessing other networks the user name and password has to be given again.