locked
Having trouble trying to get JSON info from Spiceworks Website. RRS feed

  • Question

  • For some background on what I'm trying to do, Click here - http://community.spiceworks.com/topic/295123-fun-spiceworks-community-project?page=2#entry-2035675

    Spiceworks is a web based software package. If I am logged into the website I can enter this into a browser:

    http://servername:port/api/tickets.json?total_count=true&filter=unassigned

    and I get this back (Count of currently unassigned Tickets)

    {"count":0,"filter":"unassigned"}

    When I try this:

    PS C:\> $page = (new-object net.webclient).DownloadString("http://servername:port/api/tickets.json?total_count=true&filter=unassigned")

    Exception calling "DownloadString" with "1" argument(s): "The remote server returned an error: (406) Not Acceptable."
    At line:1 char:1
    + $page = (new-object net.webclient).DownloadString("http://servername:port/api/tick ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
        + FullyQualifiedErrorId : WebException

    This may be because I'm not logged into the website, but I can't quite figure that part out, I'm trying some things now using the IE COM object, but I haven't gotten too far yet. 

    The goal is to take the above response and grab the number and use it to control a real world status indicator(already have that part figured out, just need to grab the damn number from the website).

    I'm a novice coder at best, and alot of this stuff is over my head, but I'm using this small project as a way to learn powershell and object oriented programming. I've learned alot since I've started working on it, but I've taken it as far as I can with my current knowledge. 

    Wednesday, March 13, 2013 6:58 PM

All replies

  • Error 406 indicates that is probably error in the headers sent from your client. (Ie User-Agent, Accepted mime-types etc)...

    I would recomend using Fiddler to see your request and the answer going back, a great way of troubleshooting.

    If it is as I suspect you can try and add those headers on.

    Wednesday, March 13, 2013 7:15 PM
  • Also, if you are on PowerShell 3.0, try the Invoke-WebRequest cmdlet.
    Wednesday, March 13, 2013 7:18 PM
  • I am on Powershell 3.0. I upgraded just to get the Invoke-WebRequest cmdlet. I can get the main page to return code, but I'm unsure how to pass my username and password to the website before attempting to grab the unassigned count. 

    Wednesday, March 13, 2013 7:32 PM
  • This is the login page. 

    It's from http://servername:port/login

    It doesn't show the name of the html file, so I'm assuming it's generated on the fly?

    <!DOCTYPE html>
    <!--[if lt IE 7]>  <html lang="en" class="no-js ie ie6 lte9 lte8 lte7"> <![endif]-->
    <!--[if IE 7]>     <html lang="en" class="no-js ie ie7 lte9 lte8 lte7"> <![endif]-->
    <!--[if IE 8]>     <html lang="en" class="no-js ie ie8 lte9 lte8"> <![endif]-->
    <!--[if IE 9]>     <html lang="en" class="no-js ie ie9 lte9"> <![endif]-->
    <!--[if gt IE 9]>  <html lang="en" class="no-js"> <![endif]-->
    <!--[if !IE]><!--> <html lang="en" class="no-js"> <!--<![endif]-->
      <head>
        <meta charset="utf-8" />
        <meta name="author" content="Spiceworks, Inc." />
        <meta http-equiv="X-UA-Compatible" content="IE=8" />
        <title>Spiceworks - Login Required</title>
        <link href="/stylesheets/framework.css?6101100" media="screen" rel="stylesheet" type="text/css" />
    <link href="/stylesheets/ui_elements/ui-elements.css?6101100" media="screen" rel="stylesheet" type="text/css" />
    <link href="/stylesheets/ui_elements/buttons.css?6101100" media="screen" rel="stylesheet" type="text/css" />
    <link href="/stylesheets/general.css?6101100" media="screen" rel="stylesheet" type="text/css" />
    <link href="/stylesheets/login.css?6101100" media="screen" rel="stylesheet" type="text/css" />
        <!--[if IE]><link href="/stylesheets/hacks.ie.css?6101100" media="all" rel="stylesheet" type="text/css" /><![endif]-->
    <!--[if IE 7]><link href="/stylesheets/hacks.ie7.css?6101100" media="screen" rel="stylesheet" type="text/css" /><![endif]-->
    <!--[if IE 8]><link href="/stylesheets/hacks.ie8.css?6101100" media="screen" rel="stylesheet" type="text/css" /><![endif]-->
        <script src="/assets/prototype.js?6101100" type="text/javascript"></script>
    <script src="/assets/libraries.js?6101100" type="text/javascript"></script>
        <!--[if lte IE 6]><script type="text/javascript">Browser.ie6=true;</script><![endif]-->
        <!--[if IE 7]><script type="text/javascript">Browser.ie7=true;</script><![endif]-->
      </head>
      <body class="login">
        <div id="wrapper">
          
    <div id="incompatible">
        
        <h3>Your Browser is Incompatible with Spiceworks</h3>
        <p>Either you have JavaScript, cookies, or Stylesheets (CSS) turned off or you are using a browser without the features required to use Spiceworks.</p>
        <p>Spiceworks is compatible with <a href="http://community.spiceworks.com/help/Spiceworks_Requirements#Browser" class="">modern browsers</a>, and requires JavaScript, Cookies, and Stylesheets (CSS) to function correctly.</p>
        <p>If your browser meets these requirements and you are still having issues, please send an e-mail to <a href="mailto:support@spiceworks.com">support@spiceworks.com</a> with an explanation of the problem.</p>
      
    </div>
    <script type="text/javascript">
    //<![CDATA[
    Browser.hideIncompatible();
    //]]>
    </script>
                <h1><img alt="Spiceworks" src="/images/logos/large.png?6101100" /></h1>
          <div id="message_container">
                  </div>
          <form accept-charset="UTF-8" action="/login" method="post"><div style="margin:0;padding:0;display:inline"><input name="authenticity_token" type="hidden" value="kUqoWAxjF9aLGtMN3OzkZoa+6kfZJzaQr6pbnkEhvek=" /></div><div style="margin:0;padding:0;display:inline;"><input name="_pickaxe" type="hidden" value="⸕" /></div>
      
      <p>
        <label for="user_email">Email:</label>
        
        <input autocomplete="off" class="text" id="user_email" name="user[email]" tabindex="1" type="email" />
      </p>
      <p>
        <label for="user_password">Password:</label>
        <input class="text" id="user_password" name="user[password]" size="30" tabindex="2" type="password" />
      </p>
      <p class="actions">
        <button class="sui-btn medium" data-value="login" name="btn" tabindex="4" type="submit" value="login"><span><span style="">Login</span></span></button>
        <input class="chk" id="user_remember" name="user[remember]" tabindex="3" type="checkbox" value="1" />
        <label for="user_remember">Stay logged in</label>
      </p>
    </form>
    <p id="forgot_password">
      Forgot your password? <a href="http://community.spiceworks.com/help/Resetting_Your_Password_Outside_The_Spiceworks_Desktop" class="help" target="_blank">Learn how to reset it</a>
    </p>
    <script type="text/javascript">
    //<![CDATA[
    if(!$('incompatible').visible()){
    $('user_email').focus();
    }
    //]]>
    </script>
    
          <p class="copy">
      Copyright &copy; 2006-12 <a href="http://www.spiceworks.com" class="">Spiceworks, Inc.</a> 
      "Spiceworks" and the Spiceworks logos are trademarks of Spiceworks, Inc.
      
    
      
    </p>
    
        </div>
      </body>
    </html>
    

    Wednesday, March 13, 2013 7:42 PM
  • Ok. Seems to be "working".

    I think what you need to do is to first make a post-request with the user_email and user_password.

    $formFields = @{user_email='john doe'; user_password ='123'}

    $LoginPage = Invoke-WebRequest -Method POST -Uri <loginpage> -WebSession $sessionVariable -Body $formFields -ContentType "application/x-www-form-urlencoded"

    then the next request can be for the original url where the content is.

    I noticed that there is a authenticity_token field in the form, but hopefully you can ignore that one..

    Wednesday, March 13, 2013 7:51 PM
  • Thanks Dainel,

    I will try that. I also downloaded Fiddler and am reviewing the Invoke-WebRequest help. 

    I will post my results tomorrow. 

    Wednesday, March 13, 2013 7:56 PM
  • It seems to be having a problem with the & in http://servername:port/api/tickets.json?total_count=true&filter=unassigned

    I tried quoting the & and then tried quoting the whole thing but no luck yet. 

    Wednesday, March 13, 2013 8:52 PM
  • I'm not exactly sure it's logging me in. 

    I'm using a different URL to test this now instead of the json request, just so I can see if I'm getting authenticated. I don't think that I am, it looks like the login dialog. 

    $formFields = @{user_email='username'; user_password ='password'}

    $LoginPage = Invoke-WebRequest -Method POST -Uri http://server:port/login -WebSession $sessionVariable -Body $formFields -ContentType "application/x-www-form-urlencoded"

    $Unassigned = Invoke-WebRequest http://server:port/tickets/list/my_tickets

    $unassigned

    Results:

    StatusCode        : 200
    StatusDescription : OK
    Content           : <!DOCTYPE html>
                        <!--[if lt IE 7]>  <html lang="en" class="no-js ie ie6 lte9 lte8 lte7"> <![endif]-->
                        <!--[if IE 7]>     <html lang="en" class="no-js ie ie7 lte9 lte8 lte7"> <![endif]-->
                        <!--[if IE 8]>...
    RawContent        : HTTP/1.1 200 OK
                        X-Runtime: 0
                        Cache-Control: private, max-age=0, must-revalidate
                        Date: Thu, 14 Mar 2013 13:13:07 GMT
                        ETag: "82660d2c5a1b57bc912a64eac2c3a9d1"
                        Set-Cookie: compatibility_test=testing...
    Forms             : {}
    Headers           : {[X-Runtime, 0], [Cache-Control, private, max-age=0, must-revalidate], [Date, Thu, 14 Mar 2013 13:13:07 GMT], [ETag, 
                        "82660d2c5a1b57bc912a64eac2c3a9d1"]...}
    Images            : {@{innerHTML=; innerText=; outerHTML=<IMG alt=Spiceworks src="/images/logos/large.png?6101100">; outerText=; tagName=IMG; alt=Spiceworks; 
                        src=/images/logos/large.png?6101100}}
    InputFields       : {@{innerHTML=; innerText=; outerHTML=<INPUT name=authenticity_token value=lsT2+b/uXNny2qRSapK2l4LFyAltfWIz0tCrZmoXGyI= type=hidden>; outerText=; 
                        tagName=INPUT; name=authenticity_token; value=lsT2+b/uXNny2qRSapK2l4LFyAltfWIz0tCrZmoXGyI=; type=hidden}, @{innerHTML=; innerText=; 
                        outerHTML=<INPUT name=_pickaxe value=⸕ type=hidden>; outerText=; tagName=INPUT; name=_pickaxe; value=⸕; type=hidden}, @{innerHTML=; innerText=; 
                        outerHTML=<INPUT id=user_email class=text tabIndex=1 name=user[email] type=email autocomplete="off">; outerText=; tagName=INPUT; id=user_email; 
                        class=text; tabIndex=1; name=user[email]; type=email; autocomplete=off}, @{innerHTML=; innerText=; outerHTML=<INPUT id=user_password class=text 
                        tabIndex=2 name=user[password] value="" size=30 type=password>; outerText=; tagName=INPUT; id=user_password; class=text; tabIndex=2; 
                        name=user[password]; value=; size=30; type=password}...}
    Links             : {@{innerHTML=modern browsers; innerText=modern browsers; outerHTML=<A 
                        href="http://community.spiceworks.com/help/Spiceworks_Requirements#Browser">modern browsers</A>; outerText=modern browsers; tagName=A; 
                        href=http://community.spiceworks.com/help/Spiceworks_Requirements#Browser}, @{innerHTML=support@spiceworks.com; innerText=support@spiceworks.com; 
                        outerHTML=<A href="mailto:support@spiceworks.com">support@spiceworks.com</A>; outerText=support@spiceworks.com; tagName=A; 
                        href=mailto:support@spiceworks.com}, @{innerHTML=Learn how to reset it; innerText=Learn how to reset it; outerHTML=<A class=help 
                        href="http://community.spiceworks.com/help/Resetting_Your_Password_Outside_The_Spiceworks_Desktop" target=_blank>Learn how to reset it</A>; 
                        outerText=Learn how to reset it; tagName=A; class=help; 
                        href=http://community.spiceworks.com/help/Resetting_Your_Password_Outside_The_Spiceworks_Desktop; target=_blank}, @{innerHTML=Spiceworks, Inc.; 
                        innerText=Spiceworks, Inc.; outerHTML=<A href="http://www.spiceworks.com">Spiceworks, Inc.</A>; outerText=Spiceworks, Inc.; tagName=A; 
                        href=http://www.spiceworks.com}}
    ParsedHtml        : mshtml.HTMLDocumentClass
    RawContentLength  : 4465


    Thursday, March 14, 2013 1:27 PM
  • I would guess that you need to pick up the authenticity_token field and send also when logging in. Seems to be some additional security in that field..

    As for if you have encoding issues you can use [System.Web.HttpUtility]::UrlEncode to encode parts of the URL 

    Lastly I think that if you try the transaction by browser (still using Fiddler) you will se the "correct" flow of posts and can compare to what you are getting back now. Fiddler also let's you replay and make post directly from Fiddler so you can experiment before you implement it in PowerShell.

    I found this also: http://community.spiceworks.com/scripts/show/311-spiceworks-ticket-rss-atom-feed

    • Proposed as answer by Yan Li_ Monday, March 18, 2013 3:00 AM
    Thursday, March 14, 2013 2:37 PM
  • I think I'll mess with Fiddler so I know what I'm trying to duplicate. Watching the instructional video's now. 

    Thanks alot for that link. I've searched spiceworks for weeks, never saw that one. 

    Thanks again for all your assistance so far. 

    Thursday, March 14, 2013 2:54 PM
  • Hi,

    Just checking in to see if the suggestions were helpful. Please let us know if you would like further assistance.

     

    If you have any feedback on our support, please click here .


    Cataleya Li
    TechNet Community Support

    Monday, March 18, 2013 3:01 AM
  • Daniel's Suggestions have been very helpful. I'm still working on it. I think he's right with the Auth Token, though I don't yet know what one is, but I can see it when I use fiddler. 

    I'm not a web developer, so all of this is new to me, and in order for me to solve my problem, I have to have an understanding of what's going on behind the scenes. 

    This is a pet project so I am working on it when I'm caught up on other work. I'll post my findings as soon as I make some progress. 

    Tuesday, March 19, 2013 11:41 AM
  • Hi,

    As this is not only a powershell script issue, I would like to suggest you post in the MSDN and Official scripting guys forum as well.

    Regards,

    Yan Li

    If you have any feedback on our support, please click here .


    Cataleya Li
    TechNet Community Support

    Thursday, March 21, 2013 2:25 AM
  • Hi Nextonex -

    Have you made any headway with this project?

    I'm working on the same.

    Thanks

    -Andrew

    Wednesday, November 13, 2013 5:46 PM
  • No, I had other projects and never got back to it. I would be interested in any progress you make and will continue to monitor this thread. 
    Wednesday, November 13, 2013 7:22 PM
  • This code doesn't look quite right:

    $formFields = @{user_email='username'; user_password ='password'}
    
    $LoginPage = Invoke-WebRequest -Method POST -Uri http://server:port/login -WebSession $sessionVariable -Body $formFields -ContentType "application/x-www-form-urlencoded"
    
    $Unassigned = Invoke-WebRequest http://server:port/tickets/list/my_tickets
    
    $unassigned

    What you should be doing is passing the -SessionVariable parameter to the first Invoke-WebRequest call (loading the logon page).  This is a lot like -ErrorVariable, etc; you specify a variable name without a dollar sign, and it saves the session to that variable.  Then, in future calls to Invoke-WebRequest, you use -WebSession $variable .

    You can see an example of using this technique to log on to FaceBook in the Invoke-WebRequest help file:

    (Get-Help Invoke-WebRequest -Examples).examples.Example[1]

    Wednesday, November 13, 2013 7:33 PM
  • I just posted this today:

    http://community.spiceworks.com/scripts/show/2285-log-into-spiceworks-via-powershell-for-api-access

    Also linked to from the original Spiceworks community post mentioned in the first post of this thread:

    http://community.spiceworks.com/topic/295123-fun-spiceworks-community-project?page=2

    :) -Andrew

    Wednesday, November 20, 2013 1:39 AM
  • I just posted this today:

    http://community.spiceworks.com/scripts/show/2285-log-into-spiceworks-via-powershell-for-api-access

    Also linked to from the original Spiceworks community post mentioned in the first post of this thread:

    http://community.spiceworks.com/topic/295123-fun-spiceworks-community-project?page=2

    :) -Andrew

    Interesting about the blank cookie issue.  Rather than replacing ",," with "," (which only works if there isn't more than one consecutive blank entry in the list), you could do something like this to remove all blank entries from the list:

    $cookies = $r.Headers['Set-Cookie'] -split ',' -match '.' -join ','

    $session.Cookies.SetCookies($url,$cookies)

    Edit:

    Also, on this line:

    Import-Module ($([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory()) + "System.Web.dll")
    
    # it would probably be simpler to just do this:
    Add-Type -AssemblyName 'System.Web'

    # But in fact, you don't need that assembly at all, if you're running PowerShell 3.0 or later.
    # .NET Framework 4.0 added a similar class to the System.Net assembly, and you can use it
    # without loading any other types:

    [System.Net.WebUtility]::UrlEncode($whatever)

    • Edited by David Wyatt Wednesday, November 20, 2013 1:34 PM
    Wednesday, November 20, 2013 3:36 AM
  • Thanks for the feedback.

    Creative and simple.

    -Andrew

    Monday, November 25, 2013 3:47 PM