Personal tools
You are here: Home Documentation Issue Trackers General Issue Tracker need a better script to track large files in sites

#5 — need a better script to track large files in sites

State Resolved
Area Functionality
Issue type Feature
Severity Medium
Submitted by T. Kim Nguyen
Submitted on Jan 24, 2009
Responsible T. Kim Nguyen
Return to tracker
Last modified on Nov 19, 2009 by T. Kim Nguyen
ie. on a specifiable site or on all sites, so can run periodically
Added by vanghn17 on Jan 24, 2009 01:37 PM
Might need to find a tricky way to do this since large files could also be attachments...
Added by T. Kim Nguyen on Jan 24, 2009 02:14 PM
Where can these attachments be added?

I was referring to the script

http://www.uwosh.edu/[…]/manage_workspace

and

http://www.uwosh.edu/[…]/manage_workspace

but instead of having to add them to a site then run them in place, it'd be nice to have them sit in the Zope root and be able to say "run over all the sites" or "run in just the site <XYZ>" where XYZ is presented as say a drop down list.

But the real fix is to implement file and image upload size limits, period.
Added by (anonymous) on Jan 24, 2009 02:20 PM
In attachments, I mean you can have a image inside of a News Item and such. Right now I think we're just looking at image sizes aren't we?

Also, what if some guy puts a huge image in portal_skins? That isn't indexed or anything.

Can you put those scripts on this issue(attach them) so I can have a look? I don't have access to those servers.
Added by T. Kim Nguyen on Jan 24, 2009 02:33 PM
Here's honking files
Attached:
find-honking-files.txt — text/plain, 0 Kb
Added by T. Kim Nguyen on Jan 24, 2009 02:34 PM
Here's the image one
Attached:
find-honking-images.txt — text/plain, 0 Kb
Added by T. Kim Nguyen on Jan 24, 2009 02:35 PM
Yeah these scripts probably won't see anything huge in portal_skins, nor (probably) in News, unless the recipe that Dan sent me covers those too. I'll see if I can find it.
Added by T. Kim Nguyen on Jan 24, 2009 02:36 PM
It was actually from Kurt. See below.

We'd talked about image size issues a couple of weeks ago.

There are two ways to get what you want, either use an egg
and add a part to your buildout, or copy the file to
client/etc/atcontenttypes.conf and edit it.


Egg: plone.recipe.atcontenttypes
===========================================
Here's an recipe that'll setup the config:
 http://pypi.python.org/pypi/plone.recipe.atcontenttypes/


Add a atcontenttypes-conf part to your buildout, plus
the config:

parts =
   plone
   zope2
   productdistros
   zeoserver
   client1
   client2
   zopepy
   zopeskel
   precompile
   chown
   unifiedinstaller
   fss
   atcontenttypes-conf


# And the part configs
[atcontenttypes-conf]

max-image-dimension =
   ATNewsItem:640,400
   ATImage:1024,1024

max-file-size =
   ATImage:1mb
   ATNewsItem:500kb
# ATFile:100mb


Copying the file
===========================================================
The config is based on this file:
 parts/plone/ATContentTypes/etc/atcontenttypes.conf.in

You can copy this file as 'atcontenttypes.conf' into the
same dirs as the 'client{1,2}/etc/zope.conf' files. and
make the modifications needed to max-image-dimension and
max-file-size for ATImage. Something like the following:


max-image-dimension =
   ATNewsItem:100,150
   ATImage:1024,1024

max-file-size =
   ATImage:1mb
   ATNewsItem:500kb


The stanza will look like this:

<archetype ATImage>
 # maximum file size in byte, kb or mb
 max_file_size no
 # maximum image dimension (w, h)
 # 0,0 means no rescaling of the original image
 max_image_dimension 0,0
</archetype>

# Change what you need to change.......
# ----------------------------------------
#Change the new copy to something like:
<archetype ATImage>
 max_file_size 1mb
 max_image_dimension 1024,1024
</archetype>

#And do something similar to news items if you wish.
<archetype ATNewsItem>
 max_file_size 500k
 max_image_dimension 100,125

 # enable upload of documents
 allow_document_upload yes

 <contenttypes>
   default text/html

   allowed text/structured
   allowed text/x-rst
   allowed text/html
   allowed text/plain
 </contenttypes>
</archetype>

Added by vanghn17 on Jan 24, 2009 03:44 PM
Here is a start on the script. It works fine, but you can't specify which site it will check--just does all of them right now. Should be easy enough to implement though. This script even checks portal skins.
Attached:
find-honking-stuff.py — text/x-python, 1 Kb
Added by T. Kim Nguyen on Mar 30, 2009 04:09 PM
Slight tweak to handle possible gigabyte-sized files
Attached:
findHonkingFiles.py — application/octet-stream, 1 Kb
Added by T. Kim Nguyen on Nov 19, 2009 04:14 PM
See here for the latest version of .findHonkingStuff script

http://www.uwosh.edu/plonep[…]objects-in-your-plone-sites
Added by T. Kim Nguyen on Nov 19, 2009 04:15 PM
Issue state: unconfirmedresolved
Responsible manager: (UNASSIGNED)nguyen
Installed the above recipe on all buildout Zopes or put atcontenttypes.conf into each non-buildout Zope manually.

Size limits are

image: 500 kb
image: 1024,1024

file: 30 mb
You must log in to add a response