Here's a nice HTTP wrapper that I wrote and use in my programs. Its simple and very effective. Connnection.java Code (Text): import java.io.*; import java.net.*; import java.util.*; import java.util.zip.*; import java.awt.image.BufferedImage; import javax.imageio.ImageIO; public class Connection implements Serializable { static final long serialVersionUID = 1L; String domain, referer; Map<String,String> cookies; static String rpUseragent = "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.14) Gecko/20080509 Firefox/2.0.0.14"; static String rpAcceptText = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"; static String rpAcceptPng = "image/png,image/*;q=0.8,*/*;q=0.5"; static String rpAcceptLanguage = "en-us,en;q=0.5"; static String rpAcceptEncoding = "gzip, deflate"; static String rpAcceptCharset = "ISO-8859-1,utf-8;q=0.7,*;q=0.7"; static String rpKeepAlive = "300"; static String rpConnection = "keep-alive"; static String rpContentType = "application/x-www-form-urlencoded"; public Connection( String domain, Map<String,String> cookies, String referer ) { this.domain = domain; this.cookies = cookies; this.referer = referer; } public Connection( String domain, Map<String,String> cookies ) { this( domain, cookies, null ); } public Connection( String domain, String referer ) { this( domain, new HashMap<String,String>(), referer ); } public Connection( String domain ) { this( domain, new HashMap<String,String>(), null ); } public String get( String url ) { if( url.charAt( 0 ) == '/' ) url = domain + url; try { HttpURLConnection conn = (HttpURLConnection)( new URL( url.replaceAll( " ", "%20" ) ).openConnection() ); setRequestProperties( conn ); conn.setRequestMethod( "GET" ); referer = url; return read( conn ); } catch( IOException e1 ) { e1.printStackTrace(); return null; } } public String post( String url, String[][] data ) { if( url.charAt( 0 ) == '/' ) url = domain + url; try { HttpURLConnection conn = (HttpURLConnection)( new URL( url.replaceAll( " ", "%20" ) ).openConnection() ); setRequestProperties( conn ); conn.setRequestMethod( "POST" ); conn.setDoOutput( true ); StringBuilder sb = new StringBuilder(); for( int i = 0; i < data[0].length; i++ ) sb.append( URLEncoder.encode( data[0][i], "UTF-8" ) ).append( '=' ).append( URLEncoder.encode( data[1][i], "UTF-8" ) ).append( '&' ); conn.setRequestProperty( "Content-Type", rpContentType ); conn.setRequestProperty( "Content-Length", Integer.toString( sb.length()-1 ) ); PrintWriter out = new PrintWriter( new BufferedWriter( new OutputStreamWriter( conn.getOutputStream() ) ) ); out.write( sb.substring( 0, sb.length()-1 ) ); out.close(); referer = url; return read( conn ); } catch( IOException e1 ) { e1.printStackTrace(); return null; } } public BufferedImage getImage( String url ) { try { HttpURLConnection conn = (HttpURLConnection)( new URL( ( url.charAt( 0 ) == '/' ? domain+url : url ).replaceAll( " ", "%20" ) ).openConnection() ); setRequestProperties( conn ); conn.setRequestMethod( "GET" ); conn.setRequestProperty( "Accept", rpAcceptPng ); return ImageIO.read( conn.getInputStream() ); } catch( IOException e1 ) { e1.printStackTrace(); return null; } } public boolean hasCookie( String key ) { return cookies.containsKey( key ); } public String getCookieString() { StringBuilder sb = new StringBuilder(); for( String s : cookies.keySet() ) sb.append( s ).append( '=' ).append( cookies.get( s ) ).append( ';' ); return sb.toString(); } private void setRequestProperties( HttpURLConnection conn ) { conn.setInstanceFollowRedirects( false ); conn.setRequestProperty( "User-Agent", rpUseragent ); conn.setRequestProperty( "Accept", rpAcceptText ); conn.setRequestProperty( "Accept-Language", rpAcceptLanguage ); conn.setRequestProperty( "Accept-Encoding", rpAcceptEncoding ); conn.setRequestProperty( "Accept-Charset", rpAcceptCharset ); conn.setRequestProperty( "Keep-Alive", rpKeepAlive ); conn.setRequestProperty( "Connection", rpConnection ); if( referer != null && referer.length() != 0 ) conn.setRequestProperty( "Referer", referer ); if( cookies != null && cookies.size() != 0 ) conn.setRequestProperty( "Cookie", getCookieString() ); } private String read( HttpURLConnection conn ) throws IOException { BufferedReader in = null; if( conn.getContentEncoding() == null ) in = new BufferedReader( new InputStreamReader( conn.getInputStream() ) ); else if( conn.getContentEncoding().equalsIgnoreCase( "gzip" ) ) in = new BufferedReader( new InputStreamReader( new GZIPInputStream( conn.getInputStream() ) ) ); else if( conn.getContentEncoding().equalsIgnoreCase( "deflate" ) ) in = new BufferedReader( new InputStreamReader( new InflaterInputStream( conn.getInputStream(), new Inflater( true ) ) ) ); StringBuilder sb = new StringBuilder(); String s; while( ( s = in.readLine() ) != null ) sb.append( s ).append( '\n' ); putCookies( conn.getHeaderFields().get( "Set-Cookie" ) ); return sb.toString(); } private void putCookies( List<String> cookieList ) { if( cookieList == null ) return; int index; for( String cookie : cookieList ) cookies.put( cookie.substring( 0, index = cookie.indexOf( '=' ) ), cookie.substring( index+1, cookie.indexOf( ';', index ) ) ); } } Here's how to use it: First you instantiate an instance of the Connection class like so Code (Text): Connection conn = new Connection( "http://www.neopets.com" ); Now, you can use a number of methods in the Connection class. To make a GET request, use the get( url ) method: Code (Text): conn.get( "/bank.phtml" ) Notice that you don't have to type the full URL, the domain name specified in the constructor is automatically filled in. If you do specify a domain however in the URL, it will go to that domain w/o the default domain filled in. POST requests work much the same way: Code (Text): conn.post( "/login.phtml", new String[][]{ { "username", "password", "destination" }, { user, password, "/" } } ); This is how you log into Neopets using my wrapper. Just one simple line. The POST data is stored as a 2D array. The first array is the POST attribute, the second array holds the POST data corresponding to each attribute. You may also POST using just a String, but I've never used that myself explicitly, so I won't cover that here. getImage( url ) returns an image at the given URL. This is useful for making things like ONRs. Code (Text): conn.getImage( "/captcha_show.phtml?_x_pwned=" + hash ) I find this wrapper particularly useful because you do not have to deal with referer URLs and cookies. It automatically handles both of these. It stores the previously accessed URL into a referer variable and uses that as the referer URL in the next request (see Connection.java for details). Cookies are also automatically updated, so there is no need for manually handling them. Enjoy and happy programming
+rep top java example just one point, it has been advised that you don't use a one line login sorry if you have a multiple line one coded into the wrapper, i haven't used
Yes, I just used that one particular line in the example to show how a POST request could be done with this wrapper. In fact, it wouldn't work in one line with my wrapper because otherwise it wouldn't handle the referrer URL correctly. This wrapper forces the programmer to have the program go to the correct pages in order to work. I prefer to think of each instance of a Connection as a tab or a window on Neopets. Just make sure that you use the same CookieManager for each of the Connections. Here's a full log-in example: Code (Text): conn.post( "/hi.phtml", new String[][]{ { "destination", "username" }, { "/", user } } ); String page = conn.post( "/login.phtml", new String[][]{ { "username", "password", "destination" }, { user, password, "/" } } ); // code to determine if account is frozen, etc
you couldn't post a code snippet with a complete class (ie one which works without any coding) where all it does is login could you (bogus usernames and passwords obviously)? thanks