Migrating Drupal 7 Image Files to Drupal 8 Media Images (no PHP required!)

I’m new to migrate. And it’s a beast. But I think after a few dozen hours of diligence, I think I’ve figured out my preferred method.

This post is for someone who already understands how to use views, develop custom modules, and knows how to use drush migration operations.

I originally tried using migrate_drupal, which honestly did a great job at migrating my user roles, users, and taxonomies. But the site I am migrating is 5 years old with a good amount of technical debt having been touched by over 10 (and maybe even 20) developers over its lifetime, so I think it’s time for a fresh start. I want to migrate everything else (content types, files, etc) manually.

All the Drupal 7 content types were using image (file) fields, but I want the new Drupal 8 site to use media fields. I couldn’t use the migration provided by migrate_drupal, so I had to get creative.

This will be a two step process:

  1. Use D7 views_datasource module to create JSON endpoints on my old site (and since we’re using a JSON source, you can honestly migrate from anything).
  2. Write a custom migration in D8.

Drupal 7 and Views JSON

Create a views JSON that pulls all file content with the following fields:

  • File ID (this will become the Media ID — we want to keep the IDs so when we merge content, it’ll be easier to also migrate their references)
  • Name
  • Absolute path (e.g. https://fullpath.com/sites/default/files/something.jpg)
  • Alt text
  • Date
  • User ID (Many users in CMS — I want to make sure they keep ownership of their images)
  • Title text (optional and not really used, but some people have taken the time to enter this field so I figure it would be worth migrating over)

Also, in the format settings, make sure you set a root object name (I used “images”) and leave the the “Top-level child object” field empty.


Now, your JSON object should look something like this:

{
  "images": [
    {
      "fid": "6",
      "filename": "your-image.jpg",
      "path": "https://youroldsite.com/sites/default/files/your-image.jpg",
      "alt": "People shaking hands",
      "created": "1432021142",
      "uid": "1",
      "title": ""
    },
    {
      "fid": "11",
      "filename": "your-screenshot.jpg",
      "path": "https://youroldsite.com/sites/default/files/your-screenshot.jpg",
      "alt": "Video demo",
      "created": "1432021142",
      "uid": "1",
      "title": ""
    }
  ],
  "pager": {
    "pages": null,
    "page": null,
    "count": 0,
    "limit": 0
  }
}

The Drupal 8 Migration

Now for the beast. On your new site, install the following contrib modules:

  • media (core)
  • migrate (core)
  • migrate_plus (contrib)
  • migrate_tools (contrib)

You’ll also need to apply this patch that fixes a problem with the entity_generate plugin in order to get UIDs and FIDs to import correctly into your generated file entities https://www.drupal.org/project/migrate_plus/issues/2975266

You’ll then need two files in a custom module:

  • my_module/config/install/migrate_plus.migration_group.files.yml
  • my_module/config/install/migrate_plus.migration.images.yml
# migrate_plus.migration_group.files.yml
langcode: en
status: true
module: my_module
id: files
label: File imports
description: Import files from my old website.
source_type: JSON
dependencies:
  enforced:
    module:
      - my_module
# migrate_plus.migration.images.yml
id: images
label: Files
migration_group: files
migration_tags:
  - files
source:
  plugin: url
  data_fetcher_plugin: http
  data_parser_plugin: json
  urls:
    - 'https://youroldsite.com/api/to/images'
  item_selector: images
  fields:
    -
      name: fid
      label: 'File ID'
      selector: fid
    -
      name: filename
      label: 'Name'
      selector: name
    -
      name: path
      label: 'Path'
      selector: path
    -
      name: alt
      label: 'Alt'
      selector: alt
    -
      name: created
      label: 'Created'
      selector: created
    -
      name: uid
      label: 'User ID'
      selector: uid
    -
      name: title
      label: 'Title text'
      selector: title
  ids:
    fid:
      type: integer
process:
  # Preparation.
  destination_basename:
    plugin: callback
    callable: basename
    source: path
  destination_basepath:
    plugin: default_value
    default_value: public://
  destination_path:
    plugin: concat
    source:
      - '@destination_basepath'
      - '@destination_basename'

  # Mapping.
  mid: fid
  bundle:
    plugin: default_value
    default_value: image
  name: name
  'field_media_image/title': title
  'field_media_image/alt': alt
  'field_media_image/target_id':
    -
      plugin: skip_on_empty
      method: process
      source: path
    -
      plugin: file_copy
      source:
        - path
        - '@destination_path'
    -
      plugin: entity_generate
      values:
        uid: uid
        fid: fid
  status:
    plugin: default_value
    default_value: 1
  created: created
  changed: created

destination:
  plugin: entity:media

Next Steps

Install the your custom module, and you should be good to go with a drush mim images.

I’m going to use this to also migrate PDFs and other documents. I thought about creating one huge migration to migrate all file entities to all media entities, but I figure it would be cleaner (and more maintainable) to do one bundle at a time.

This took me longer than I’d like to admit to get together, but hopefully this works for you like it did for me. Feel free to drop a comment for any comments/questions. Good luck!