perl

  • strict warning: Non-static method view::load() should not be called statically in /var/www/freeasinkittens.com/sites/all/modules/views/views.module on line 906.
  • strict warning: Declaration of views_handler_argument::init() should be compatible with views_handler::init(&$view, $options) in /var/www/freeasinkittens.com/sites/all/modules/views/handlers/views_handler_argument.inc on line 0.
  • strict warning: Declaration of views_handler_filter::options_validate() should be compatible with views_handler::options_validate($form, &$form_state) in /var/www/freeasinkittens.com/sites/all/modules/views/handlers/views_handler_filter.inc on line 0.
  • strict warning: Declaration of views_handler_filter::options_submit() should be compatible with views_handler::options_submit($form, &$form_state) in /var/www/freeasinkittens.com/sites/all/modules/views/handlers/views_handler_filter.inc on line 0.
  • strict warning: Declaration of views_handler_filter_node_status::operator_form() should be compatible with views_handler_filter::operator_form(&$form, &$form_state) in /var/www/freeasinkittens.com/sites/all/modules/views/modules/node/views_handler_filter_node_status.inc on line 0.
  • strict warning: Non-static method view::load() should not be called statically in /var/www/freeasinkittens.com/sites/all/modules/views/views.module on line 906.
  • strict warning: Declaration of views_plugin_style_default::options() should be compatible with views_object::options() in /var/www/freeasinkittens.com/sites/all/modules/views/plugins/views_plugin_style_default.inc on line 0.
  • strict warning: Declaration of views_plugin_row::options_validate() should be compatible with views_plugin::options_validate(&$form, &$form_state) in /var/www/freeasinkittens.com/sites/all/modules/views/plugins/views_plugin_row.inc on line 0.
  • strict warning: Declaration of views_plugin_row::options_submit() should be compatible with views_plugin::options_submit(&$form, &$form_state) in /var/www/freeasinkittens.com/sites/all/modules/views/plugins/views_plugin_row.inc on line 0.

Filestore2 files recovered

Finally after being lost in limbo since upgrading to drupal 5, I've managed to recover the old files uploaded to the radicalreference.info website.

Drupal is a great tool, but I've learned the hard way that you really have to be careful what contributed modules you rely on. 4 years ago when I first built the radicalreference.info site in drupal 4.5, I decided to use a module that was not long for this world. after upgrading from 4.5 to 4.6 (where filestore became filestore2) and then to 4.7, the filestore2 and fscache modules ceased development. This left us with a ton of data that was not accessible (ok not a ton, but 77 nodes/files from the early days of the project).

At last weekend's NYC Drupal Camp #5, Joe Golden and I started working on a perl script that would recover the files and move the nodes from filestore2 to standard drupal page nodes with file attachments.

What follows is the end result, it's also attached to this post if you want to download it and use it.

Thanks Joe for the motivation to finally get this task out of the way!

#!/usr/bin/perl
#
#
# This is for converting filestore2 nodes/files into attachments to page nodes
#
# Use at your own risk
#
# written to recover files for the RadicalReference.info website
# by Joe Golden and Eric Goldhagen
# code licensed under:
# GNU GENERAL PUBLIC LICENSE
# Version 2, June 1991

# connect to database and format results
use DBI;
#DATA SOURCE NAME
my $dsn = "dbi:mysql:databasename:localhost:3306";
# PERL DBI CONNECT
my $dbh = DBI->connect($dsn, "username", "password");
#initialize various variables
my $number_changed="0";
my $starting_revision="0";
my $starting_revisions="";
my $howmanyfiles="0";

#directories relative to location of this script
#set the path to fscache directories
my $oldpath="misc/fscache/fscache/";
#set path to dir drupal uploads files to now
my $newpath="documents/";

#get starting number for fid
my $starting_fids="";
my $starting_fid="";
$sth=$dbh->prepare("select id from sequences where name='files_fid'");
$sth->execute();
$starting_fids=$sth->fetchrow_hashref();
$starting_fid=$starting_fids->{'id'};
$starting_fid=$starting_fid+1;

#select information to be used in moving files
print $starting_revision."--start at\n\n";
$sth2= $dbh->prepare("select filestore2.nid, fscache.filename, fscache.cachename, fscache.mimetype, fscache.size from filestore2 join fscache on filestore2.fsid = fscache.fsid");
$sth2->execute();
#loop through results
while (@data = $sth2->fetchrow_array()) {

my $filename, $path_stored, $realpath;

my $nid=$data[0];
my $file=$data[1];
my $cachename=$data[2];
my $filemime=$data[3];
my $filesize=$data[4];

$filename = $file;
$path_stored = $cachename;
if ( $path_stored =~ /(..)(..)(....)/ ){
$realpath = $oldpath.$1."/".$2."/".$path_stored;
# print "real path is $realpath\n";
} else {
print "badd pattern". $nid."\n";
}
my $cleanfilename = $filename;
$cleanfilename =~ s/ /\\ /go;
my $filemover= "cp ".$realpath." ".$newpath.$cleanfilename;
my $newfilepath=$newpath.$filename;
print "file: ".$realpath." moved to ".$newpath.$filename."\n";
`$filemover`;

#increment the counter for files
$howmanyfiles=$howmanyfiles+1;

#db query to insert a new file attachment
#for the right node
my $constructed_statement ='insert into files (fid,nid,filename,filepath,filemime,filesize) values ('.$starting_fid.','.$nid.',"'.$filename.'","'.$newfilepath.'","'.$filemime.'","'.$filesize.'")';
$sth = $dbh->prepare($constructed_statement);
$sth->execute();

#db query for update of file_revision

#need vid of node for vid here
my $nodevids="";
my $nodevid="";

my $nodevidstatement="select vid from node where nid=".$nid;
$sth=$dbh->prepare($nodevidstatement);
$sth->execute();
$nodevids=$sth->fetchrow_hashref();
$nodevid=$nodevids->{'vid'};

my $constructed_statement='insert into file_revisions (fid,vid,description,list) values ('.$starting_fid.','.$nodevid.',"",1)';
$sth=$dbh->prepare($constructed_statement);
print $constructed_statement."\n";
$sth->execute();

print "attached to node:". $nid." revision:".$nodevid."\n\n";

#increment the fid
$starting_fid=$starting_fid+1;
}

#db query change info for node type after we're done moving everything
$sth=$dbh->prepare('update node set type="page" where type="filestore2"');
$sth->execute();

#insert our new node fid into sequence table

my $constructed_statement='UPDATE sequences set id='.$starting_fid.' where name="files_fid"';
$sth=$dbh->prepare($constructed_statement);

print "-----------------------------------\n";
print "final fid is ".$starting_fid."\n";
$sth->execute();
print $howmanyfiles." files converted\n\n";
print "please check the output above for any errors that might have happened\n\n";

DBI->disconnect_all();
exit;

Thank you!

Hi Eric - I can't believe you just posted this. I am in the process of upgrading a client from Drupal 4.6 -> 6 and just realized that we have a big issue with converting from filestore2. Your script seems to be the solution. I'll give it a try and let you know how it goes. Thank you!

let me know how it goes!

good luck, let me know if anything is confusing or needs better documentation. Note that the script is written explicitly against the mysql database schema for drupal 5, so run it when you are at that point of the upgrade.

Worked

Hi,

Just to let you know, I finally made the time to get back to this upgrade and your script worked perfectly. Thanks for pointing out that it needed to run against Drupal 5, as I was originally planning on running it at 4.7.

Thanks again!
-Courtney

Filestore2 files recovered

Hi,

Your post has helped me to recover my old files. Thanks for the script.