Attachments and Large Packets - a better solution?

One common implementation problem we're running into is inline attachments for lixi packets.

The schema defines an element to base64 encode binary files. Typically, this runs from 1-6mb in packet size for a completed valuation report (PDFs, titles, invoices, etc).

<xs:element name="InlineAttachment" type="xs:base64Binary">

What tends to happen is someone deploys a system, and a few days later, packets are being rejected as they are beyond the capabilities of the server's default configuration.

We'd like to get opinions on how others have solved this, and also propose an idea or two.

The suggestion:
So, a lot of valuation systems are using SOAP and are doing so from web based systems.

Common tools for linux environments include things like 'wget' and there are an abundance of simple HTTP client libraries in a variety of languages.

HTTP also provides a mechanism to do authentication (HTTP Basic Authentication), as well as compression ("Accept-Encoding").

I'd like to suggest:

  <xs:element name="Attachment">
    <xs:complextype>
      <xs:sequence>
        <xs:element minoccurs="1" ref="Identifier">
        <xs:element minoccurs="1" ref="RelatedEntityRef">
        <xs:element minoccurs="1" ref="InlineAttachment">
      </xs:element>
      <xs:attribute name="SourceDomain" type="xs:string">
      <xs:attribute name="Filename" type="xs:string">
    </xs:attribute>
  </xs:attribute>
  1. Loses the minOccurs=1 restriction on InlineAttachment
  2. Either the Identifier element or a specific new element is used to capture a URL
    <Identifier uniqueid="http://vx.valex.com.au/documents/view.php?id=1">

    or

    <File href="https://vx.valex.com.au/documents/view.php?id=1" type="image/jpeg">
  3. Implementors determine an authentication mechanism between the systems, based on HTTPS and HTTP Basic Authentication

    A simple client would:

    1. Match the domain against an internal list of usernames/passwords to use
    2. Send a HTTP GET, and provide the username/password
    3. If needed, the simple client specify it can accept gzip/other compression encoding to the server (just like your browser does)
    4. If needed, the simple client respect last-modified-times / e-tags (also like your browser)

So:
What do people think of such a thing?
How have others been dealing with this situation?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

"But the problem with this

"Also, we've found on the first go around, no one updates their server configuration the required amount."

Why not send new valfirms an email asking them to increase the server's max request size and get a confirmation back? Its a lot simpler and would save extra development time for everyone too.

Or give Valfirms a choice on which method to use, since one method is already in place.

"Why not send new valfirms

"Why not send new valfirms an email asking them to increase the server's max request size and get a confirmation back? Its a lot simpler and would save extra development time for everyone too."

Well, we do, in the exact same manner we've sent yourselves emails asking to make sure all suburb information on sales records is accurate against Australia post.

It gets complicated in both situations when there's a third party involved who doesn't see the same urgency or need - ie, the commercial data providers or a hosting provider.

Are you tuned into the discussion on this via the LIXI mailing list?

Re: Why not send new valfirms

The thing is something like a config change on the server is a simple task to perform and only takes a few minutes - so this can be readily done. Whereas the something like postcodes on legacy sales data is a significant problem that requires time and resources to solve - this cannot be readily done.

Another difference is that if the packet size is not increased, it can be a show-stopper that can cause an entire job to fail. Whereas inconsistencies with sales data does not pose the same issue.

No the developers are not on the LIXI mailing list.

Why not allow for bigger attachment sizes?

Why not allow for bigger attachment sizes on the server and document this for the affected parties? Seems like a much simpler solution to me. But the problem with this approach is that there will be a lot more web traffic. However one could argue the same traffic would occur in a 'delayed get' solution as well, as the same files need to retrieved at some point in time.

Rasika.

"But the problem with this

"But the problem with this approach is that there will be a lot more web traffic."

base64 encoding tends to increase file size 1.3 times, HTTP allows compression

Also, we've found on the first go around, no one updates their server configuration the required amount.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.