22 replies to this topic

[Fabien Royer]

Hi,

I wrote a non-recursive, low-overhead JSON parser intended to run in conditions where memory is tight.
The parser supports reading from a string in memory or from a file on an SD card for example.

The source code and unit tests (see \Samples\JSONParserTest) are located here: http://netduinohelpe...list/changesets

Cheers,
-Fabien.

[Stefan]

Oh nice!

[Mario Vernari]

Fabien, have you any comparison (perf/mem) with the Xml? I used the Xml (tiny) to exchange data from N+ to Silverlight, but it was pretty heavy. It would be interesting to compare side-by-side. Good job, anyway! Cheers

[Fabien Royer]

Hi Mario,

I have no benchmark / comparison with XML parsing in terms of performance / memory usage: in order to compare apples to apples, I'd have to write an XML parser using the same architecture really. I think I'll leave that as an exercise to the reader

My goal was to have a stand-alone, lightweight JSON parser, suitable for Netduino/Plus applications: since I could not find one on the InterWebs, I wrote one.

Cheers,
-Fabien.

[evtol]

Hi Fabien!

First of all thanks for this lib I found it very useful!

But I have a little problem. I get an System.ArgumentOutOfRangeException when I try to parse a string that has Unicode characters like "email": "user\u0040gmail.com".

Do you know what might cause this problem?

Thanks

[Fabien Royer]

Hi evtol, Thanks. I'm glad that you find it useful I don't know what the problem might be yet because I tested with Unicode characters in strings. Let me take a look and I'll get back to you on this thread. Cheers, -Fabien.

[Fabien Royer]

Ok. I found the issue and checked-in the fix on CodePlex. Can you please give it a try and confirm that everything is OK now? Cheers, -Fabien.

[evtol]

Ok. I found the issue and checked-in the fix on CodePlex.
Can you please give it a try and confirm that everything is OK now?

Cheers,
-Fabien.

Yes it works perfect now!!! Thanks!

[Fabien Royer]

Awesome! Thanks for circling back

[bulletprooffool]

Awesome! Thanks for circling back

hi Fabien,

this is exactly what I need - though I am battling to figure out how to actually send the JSON request and capture the return.
I can manage normal HTTP requests - but of course going the following (.Net 4) route for fetching ia a no go:


public static void WebRequestinJson(string url, string postData)
{
StreamWriter requestWriter;

var webRequest = System.Net.WebRequest.Create(url) as HttpWebRequest;
if (webRequest != null)
{
webRequest.Method = "POST";
webRequest.ServicePoint.Expect100Continue = false;
webRequest.Timeout = 20000;

webRequest.ContentType = "application/json";
//POST the data.
using (requestWriter = new StreamWriter(webRequest.GetRequestStream()))
{
requestWriter.Write(postData);
}
}
}

Do you have any examples of sending a JSON request from a Netduino as a client?

Thanks

[Fabien Royer]

hi Fabien,

this is exactly what I need - though I am battling to figure out how to actually send the JSON request and capture the return.
I can manage normal HTTP requests - but of course going the following (.Net 4) route for fetching ia a no go:


public static void WebRequestinJson(string url, string postData)
{
StreamWriter requestWriter;

var webRequest = System.Net.WebRequest.Create(url) as HttpWebRequest;
if (webRequest != null)
{
webRequest.Method = "POST";
webRequest.ServicePoint.Expect100Continue = false;
webRequest.Timeout = 20000;

webRequest.ContentType = "application/json";
//POST the data.
using (requestWriter = new StreamWriter(webRequest.GetRequestStream()))
{
requestWriter.Write(postData);
}
}
}

Do you have any examples of sending a JSON request from a Netduino as a client?

Thanks

Hi there,

I don't have a ready-made example to give you because I haven't had a need to do this yet ;-)

However, you should consider the following:

• The Netduino Plus has 28KB of RAM and using the System.Net classes for Web requests will generally result in out of memory exceptions. If you're trying to serialize data into JSON on top of that, your mileage will be further reduced... You should consider implementing a lightweight socket client to perform HTTP requests. See the KloutKlock sample application part of the netduino helpers library to see how to do that.
• Serializing to JSON: implement a simple serializer for the data types that you need to support and serialize to JSON format straight to a file on the SD card. Then, post from the serialized data on the SD card to the web service.
• Both of these methods will ensure that your application will remain stable and will not run out of memory.

Does this make sense?

[NeonMika / Markus VV.]

Hi Fabien, thank you for your JSON parser, great piece of code. I saw your netduino helpers on Codeplex. Are there similarities to the NETMF Toolbox? Some code pieces reminded me of it. I just had the idea to write a litte converter for different data types to JSON to be used on the N+ to send data over the network. But I haven't worked with JSON before, is it hard to understand how to convert data to JSON? Greets, Markus.

[bulletprooffool]

Hi there,

I don't have a ready-made example to give you because I haven't had a need to do this yet ;-)

However, you should consider the following:

• The Netduino Plus has 28KB of RAM and using the System.Net classes for Web requests will generally result in out of memory exceptions. If you're trying to serialize data into JSON on top of that, your mileage will be further reduced... You should consider implementing a lightweight socket client to perform HTTP requests. See the KloutKlock sample application part of the netduino helpers library to see how to do that.
• Serializing to JSON: implement a simple serializer for the data types that you need to support and serialize to JSON format straight to a file on the SD card. Then, post from the serialized data on the SD card to the web service.
• Both of these methods will ensure that your application will remain stable and will not run out of memory.

Does this make sense?

Thanks Fabien,
I figured system.Net was not the way to go and have been playing with the NetMF WebRequest instead.
The page I need to collect is a simple 4k xhtml - so I could write a very basic XML parser (the tags etc will be consistent)

system.XML is of course too big for the NetduinoPlus too - so I was hoping you had an example where you were using your JSON parser with a simple web request that worked on the Netduiino (I assumed that you already had this code)

Thanks for the reply - it was infact helpful and has pointed me in a new direction (did not consider using the SD for storage)
The clases in KloutKlock definitely open some new options - so I'll see how i get on and post back - thanks for the help

[Coding Smackdown]

Thanks Fabien,
I figured system.Net was not the way to go and have been playing with the NetMF WebRequest instead.
The page I need to collect is a simple 4k xhtml - so I could write a very basic XML parser (the tags etc will be consistent)

system.XML is of course too big for the NetduinoPlus too - so I was hoping you had an example where you were using your JSON parser with a simple web request that worked on the Netduiino (I assumed that you already had this code)

Thanks for the reply - it was infact helpful and has pointed me in a new direction (did not consider using the SD for storage)
The clases in KloutKlock definitely open some new options - so I'll see how i get on and post back - thanks for the help

I've gone down the route your suggesting and even with a bare bones XML parser you'll be bouncing up against the memory constraints of the device. If you can get something even simpler to parse you'll be much better off.

Sometimes even though you love a device it may not be the right thing for the job and you may need to pick a device that has the capabilities you need. I originally wanted to use the Netduino Plus for a print agent that would pull XML from a service and dump it out to a receipt printer, but by the time I got web request and xml parsing coded I didn't have enough resources to do the IP based printing.

If you really want to use the device you may have to rethink how you'll get the data and process it in order to keep within the constraints of the device.

Regards,

Jim Lavin

[Msbh]

Hey fabian I cannot find parser on code plex or might codeplex wont let me access it. can u please mail me malikmsbh@gmail.com and please elobrate how to use it also little bit bit new . I need it urgently Thanks

[NaiNyshAH]

Hi Sir, I just download your code related to json parser but when i start testing it for each function it gives exception like stack empty. can you please tell that whats the issue. regards Hussnain

[Fabien Royer]

Hi Hussnain, That parser code is broken and I have yet to fix it... Sorry. Please take a look at one of the previous versions checked in the repository for a version that is known to work within some limitations. I made a note of this on the project documentation page. Cheers, -Fabien.

[Alex531]

Any updates on when the current JSON parser codeset will be fixed? Thanks, Alex

[Fabien Royer]

Hi Alex, No updates: my focus is entirely on Go! module development at the moment. Contributions are welcome though Cheers, -Fabien.

[Dave M.]

Hi everyone, just wondering if there is a status update on this project. I was looking around for a JSON parser for the N+2 and this is only one of two projects that I have found so far. I have confirmed that the latest code available fails all of the tests. Because it still looks like Fabien doesn't have time to fix the parser, I started to work on it yesterday. I am making some progress, but while I understand exactly what Fabien's design intends to do, there are little gotchas here and there that trip me up. Besides saying "thank you" for providing us with the code, I have to say "THANK YOU" for providing unit tests with this project, Fabien. That's the only way I'll be able to verify that things are working. So far, I have the first two unit tests working. I'm now ironing out problems with the serializing of arrays of strings in the third test case. And one of the outstanding issues is that my modifications to the Find() method that has the ArrayList outparam unintentionally made it output an ArrayList of JSONObjects and not Hashtable, so I temporarily changed the unit tests to take this into account by casting in the loop.