Connect and access sharepoint webservice from java

There are numerous articles over the internet about different approaches to access sharepoint from variety of languages like .Net, java, Javascript, Ruby etc. But these articles are in Bits and Pieces. No where they have completely running code which someone can use OTS.
There is a open source wrapper library (java-sharepoint-library) to access these webservices, but its extremely buggy, I do not know, if that project is being maintained anymore. Here is the link, java-sharepoint-library , knock yourself out.

And here it is, my version of simple program, which authenticates , connects, accesses, Check-Out and Checks-In, Downloads/Uploads a given document.

Credits to various generous people out there who has helped lot of people with thier bits.

Let's Start

Step 1. Download the wsdl files from the sharepoint.

Download all WSDLs from your sharepoint. the URL should look like the one in example.

Alerts.wsdl, dspsts.wsdl, lists.wsdl, publishedlinksservice.wsdl, sites.wsdl, versions.wsdl
admin.wsdl, dws.wsdl, meetings.wsdl, permissions.wsdl, socialdataservice.wsdl, views.wsdl
authentication.wsdl, forms.wsdl, people.wsdl, sharepointemailws.wsdl, spsearch.wsdl, webpartpages.wsdl
copy.wsdl, imaging.wsdl, search.wsdl, sitedata.wsdl, usergroup.wsdl, webs.wsdl

For example:  [Download all WSDLs]
curl https://mysharepointsite.com/_vti_bin/Lists.asmx?WSDL > lists.wsdl
curl https://mysharepointsite.com/_vti_bin/Versions.asmx?WSDL > versions.wsdl

Step 2. Generate Java Stubs for these wsdls

In order to use Sharepoints Web Services with Java, you will need to generate its stub classes with its WSDLs. Use the JDK wsimport.exe for this.

For example:  Generate Java Stubs for these wsdls
“<pathtoJDK>\wsimport.exe” -p "com.microsoft.schemas.sharepoint.soap" -keep -extension lists.wsdl
“<pathtoJDK>\wsimport.exe” -p "com.microsoft.schemas.sharepoint.soap" -keep -extension versions.wsdl

Do this for all the WSDLs you downloaded. and Import those classes in your project.

Step 3. Place the WSDLs in your resources

For some reason the wsimport.exe hardcodes the path to the wsdl files from your local disk. To avoid this you will have to place the wsdl files in your project and give resources as your buildpath.
1. Make a directory "Resources" in your project, copy WSDL directory in it.
2. In Eclipse, Goto Project -> Properties -> Java Build Path -> Sources Tab, and add the Resources folder there.
Now all your files under resources directory will be loaded at runtime.

Or you can follow the second approach given in TUG'S BLOG

Step 4. Now write the actual program to connect

Credits goes to David, but there were tweaks which I had to google and finally come up with this working version

Step 4.1 Authenticate

[Authenticate the client and get the stub for lists ws] (SPClient.java)
  public static ListsSoap getSPListSoapStub(String username, String password, String url) throws Exception {
      ListsSoap port = null;
      if (username != null && password != null) {
          try {
              // This is to avoid the error where it tries to find wsdl file, 
              // due to hardcoded path of wsdl in your stub generated by wsimport.exe
              //
              // Somehow class.getResource() did not work for me, 
              // so I am using class.getClassLoader.getResource()
              //
              URL wsdlURL = new URL(getInstance().getClass().getClassLoader().getResource("wsdl/lists.wsdl").toExternalForm());
              Lists service = new Lists(wsdlURL);
              port = service.getListsSoap();
              ((BindingProvider) port).getRequestContext().put(BindingProvider.USERNAME_PROPERTY, username);
              ((BindingProvider) port).getRequestContext().put(BindingProvider.PASSWORD_PROPERTY, password);
              
              //To avoid, error with endpoint not supported, you need to give end point url here.
              //
              URL convertedurl = convertToURLEscapingIllegalCharacters(url+"/_vti_bin/Lists.asmx");
              ((BindingProvider) port).getRequestContext().put(
                      BindingProvider.ENDPOINT_ADDRESS_PROPERTY, convertedurl.toString());
           
          } catch (Exception e) {
              e.printStackTrace();
              throw new Exception("Error: " + e.toString());
          }
      } else {
          throw new Exception("Couldn't authenticate: Invalid connection details given.");
      }
      return port;
  }

Convert the URL characters, to avoid errors in endpoint url. Coz the code above does not identify spaces in URL
For eg: if you would have URL like http://mysharepoint.com/Book names , then you will get unsupported URL exception
So you need to set endpoint URL as http://mysharepoint.com/Book%20names

[Convert the URL characters] (SPClient.java)
  public static URL convertToURLEscapingIllegalCharacters(String string){
      try {
          String decodedURL = URLDecoder.decode(string, "UTF-8");
          URL url = new URL(decodedURL);
          URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef()); 
          return uri.toURL(); 
      } catch (Exception ex) {
          ex.printStackTrace();
          return null;
      }
  }

Step 4.2 Display sharepoint list

You can display the list by giving the list name. Here I assume a list named "Documents" to be retrieved. Otherwise you can use an ID also, which can be accessed.

[Display sharepoint root list] (SPClient.java)
  /**
   * Connects to a SharePoint Lists Web Service through the given open port,
   * and reads all the elements of the given list. Only the given column names
   * are displayed.
   */ 
  public static void displaySharePointList() throws Exception {
          try {


              // you can also give id of "Documents" node
              //{44131435-EAFB-4244-AA39-F431F55ADA9B}
              //
              String listName = "Documents";
              String rowLimit = "150";
              
              //Columns names to extract
              //
              ArrayList<String> listColumnNames = new ArrayList<String>();
              listColumnNames.add("LinkFilename");
              listColumnNames.add("FileRef");

              
              //Here are additional parameters that may be set
              String viewName = "";
              GetListItems.ViewFields viewFields = null;
              GetListItems.Query query = null;
              GetListItems.QueryOptions queryOptions = null;
              String webID = "";
               
              //Calling the List Web Service
              GetListItemsResponse.GetListItemsResult result = listsoapstub.getListItems(listName, viewName, query, viewFields, rowLimit, queryOptions, webID);
              Object listResult = result.getContent().get(0);
              if ((listResult != null) && (listResult instanceof Element)) {
                  Element node = (Element) listResult;

                  //Dumps the retrieved info in the console
                  Document document = node.getOwnerDocument();
                  LOGGER.info("SharePoint Online Lists Web Service Response:" + SPClient.xmlToString(document));

                  //selects a list of nodes which have z:row elements
                  NodeList list = node.getElementsByTagName("z:row");
                  LOGGER.info("=> " + list.getLength() + " results from SharePoint Online");

                  //Displaying every result received from SharePoint, with its ID
                  for (int i = 0; i < list.getLength(); i++) {

                      //Gets the attributes of the current row/element
                      NamedNodeMap attributes = list.item(i).getAttributes();
                      LOGGER.info("******** Item ID: " + attributes.getNamedItem("ows_ID").getNodeValue()+" ********");

                      //Displays all the attributes of the list item that correspond to the column names given
                      // Note the column names prepended with "ows_" coz internal name format is that.
                      //
                      for (String columnName : listColumnNames) {
                          String internalColumnName = "ows_" + columnName;
                          if (attributes.getNamedItem(internalColumnName) != null) {
                              LOGGER.info(columnName + ": " + attributes.getNamedItem(internalColumnName).getNodeValue());
                          } else {
                              throw new Exception("Couldn't find the '" + columnName + "' column in the '" + listName + "' list in SharePoint.\n");
                          }
                      }
                  }
              } else {
                  throw new Exception(listName + " list response from SharePoint is either null or corrupt\n");
              }
          } catch (Exception ex) {
              ex.printStackTrace();
              throw new Exception("Exception. See stacktrace.\n" + ex.toString() + "\n");
          }
}

Step 4.3 Setup the default authentication

[Setup the Default authentication] (SimpleAuthenticator.java)
import java.net.Authenticator;
import java.net.PasswordAuthentication;

public class SimpleAuthenticator extends Authenticator {

  private final String username;
  private final char[] password;

  public SimpleAuthenticator(final String username, final String password) {
    super();
    this.username = new String(username);
    this.password = password.toCharArray(); 
  }

  @Override
  public PasswordAuthentication getPasswordAuthentication() {
    return (new PasswordAuthentication (username, password));
  }
}

//Use class in main
//
SimpleAuthenticator authenticator = new SimpleAuthenticator(username, password);
Authenticator.setDefault(authenticator);

Another approach given at derekjmiller blog but i Did not use it.

Step 4.4 Putting it all together

Here I show, how to display sharepoint list. If you need to know other functions, have a look at this gist.
SharePointClient

[SharePointClient] [https://gist.github.com/TheAshwanik/983fde4be42022ac1f29] (SharePointClient.java) 
public class SPClient {
  
  private static String username = "your sharepoint username";
  private static String password = "your sharepoinnt password";
  private static String BasesharepointUrl = "https://mysharepoint.com/Book Names";
  private static ListsSoap listsoapstub;
  private static VersionsSoap versionssoapstub;
  private static CopySoap copysoapstub;
  
  private static SharePointClient getInstance(){
      return(new SharePointClient());
  }
  public static void main(String[] args) {
      try {
          
          NtlmAuthenticator authenticator = new NtlmAuthenticator(username, password);
          Authenticator.setDefault(authenticator);
          
          //Authenticating and Opening the SOAP port of the Copy Web Service
          listsoapstub = SharePointClient.getSPListSoapStub(username, password, BasesharepointUrl);
      
          // Displays the lists items in the console
          SharePointClient.displaySharePointList();
      
      } catch (Exception ex) {
          ex.printStackTrace();
          System.err.println(ex);
      }
  }

That's it for now friends. Hope you guys will not struggle for this info.

Cheers :)

Resources