For a while, I have been looking (googling) for a way to serve up precompressed files. I have the server configured to compress files on the fly; but I knew there was a way to precompress files. Since I’m not an apache url rewriting expert, this one eluded me.
However, I just ran across a talk on MySql performance and scalability (take a read it has some good real world knowledge). The gem for me was at the end of the presentation on the last slide.
AddEncoding gzip .gzip
# If the user accepts gzip data
RewriteCond %{HTTP:Accept-Encoding} gzip
# … and we have a .gzip version of the file
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME}.gzip -f
# then serve that instead of the original file
RewriteRule ^(.*)$ $1.gzip [L]I have googled for that for far too long. It’s simple now that I see it, I just wasn’t able to put it all together for some reason. Guess I was expecting something more complicated. Anyway, I’m jotting it down here so I don’t forget (or the link goes away for some reason). I want to implement this real soon.
It is not possible to have an anonymous Put to S3 and a private Get. Turns out the anonymous user owns the Object. Therefor any anonymous user (which is everyone) can read the file. That’s bad. If you figure out a way for anonymous puts with private gets, please leave a comment.
I was able to figure out a slightly different mechanism that works almost as well as a PUT/GET. Oh – I don’t want to do an authenticated Put as I don’t want to have to hit the application server for every file piece (I am uploading in multiple chunks as I believe the browser will timeout the http connection – need to figure out the timeout limits though) and I don’t want to have to include the S3 secret keys in the client code (both options don’t appeal as a solution for this use case).
For background, I am creating a file upload utility for a client. They want to provide the ability for their clients to upload 700MB – 1.4GB data files with on eye towards even larger files. I want to make sure the client has the best experience possible, I realize large files for some clients can be a burden given the current state of upload bandwidth for most people. However, upload speeds are rapidly changing. At 2Mbps a 1.4GB file takes….1 hour 40 minutes. Now on my personal puny link of 256Kbps it takes 13 hours for the upload (here is a handy calculator). So that the receiving bandwidth is not an issue and can support multiple clients concurrently, the upload is being outsourced to S3 for now.
Anyway, I have to create a mechanism to get the files to S3. There are lots of moving parts for the application, one of the trickiest was the actual upload connection. I tried several techniques. Turns out amazon has a Form Post protocol that can be utilized. The important part is to create a “policy” in S3 parlance that is signed. The policy contains some specifics about the upload and an expiration.
Once I realized what to do it took a little playing around to create the correct html form post (I’m actually mimic it; but it looks the same on the receiving end). There is also a great utility to help create the policy – actually it was a big time saver for the prototype. Check it out on the amazon site: Policy Creator. It’s not pretty; but works really well.
The docs for the Post Protocol are in the Amazon Documentation section. Check them out and read carefully.
Chris….
Today we officially turned off all SSL2 support so that we could be PCI compliant. After this we passed the certification scan. Yeah. We bit the bullet and decided that SSL2 has been deprecated for years and the versions of IE6 we could find on different computers all supported SSL3. So, while there isn’t much information out there about SSL2, we will take that as a sign that we will not have an issue. Besides, everyone else who is PCI compliant will have the same issue.
If you want to setup your web servers to be PCI compliant, these are the settings we used:
SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM
These will ensure that only SSL3 and TLS1 are used with Medium to High Cipher protocols. Also, all “not secure” ciphers are expressly excluded from the acceptable list.
Here is a tool we used to scan our web servers to verify which protocols are being used (much easier to use this then to request a PCI scan from our vendor in order to verify this changed worked).
SSL Digger by Foundstone
We are in the process of having one of our applications certified to be PCI compliant. On our first attempt, we had only a handful of items to address. Most of them we were able to correct in an afternoon. The last outstanding one deals with how our SSL is configured. Basically, the PCI standard does not allow for SSL2. This is easy enough to correct with some directives in the web server configuration. The issue that I am trying to research is what affect this will have on browsers.
Specifically, does IE 6 support SSL3. Or do some of the IE6 version not suport it. The application is a consumer application that uses SSL and we want to make sure we are not excluding some users because the are on an older browser. Time to open up google and do some research.