Managing development and production assets with Gulp

What is a task-runner?

In this article I assume that you have basic knowledge about task-runners e.g. Gulp or Grunt. For newbies I'll only explain it briefly - they are tools for performing automated, programmable jobs. For example Semantic-UI framework, which I described earlier has build tools containing tasks for installing, combining, compiling assets. After running one command it creates four distribution files from several source files:

semantic.css // styles file after compiling Less source
semantic.min.css // minified version
semantic.js // JavaScript file which combines all JS plugins
semantic.min.js // minified version

Gulp has a lot of plugins available to download via npm, they can make some amazing things and are really well documented: http://gulpjs.com/plugins/

Why should I need a task-runner?

First of all - automation. Thanks to Gulp we can make our developer jobs easier, repetetive tasks like running unit tests, compiling, minification, copying, deleting, initial installation of the project - all can be automated!

One of the problems I stumbled upon during one project was to keep different assets on developer and production instances. For example on local machine, developer would like to keep separate JavaScript and CSS files for debugging and for production we need minifed and concatenated versions. Compiling those assets each time we deploy to production may be time-consuming, Gulp is there to help!

What is actually the problem?

In our example we are going to manage client application written in AngularJS, which consists of HTML, JavaScript and LESS/CSS files.

We will do the following:

  1. Use npm to manage NodeJS modules
  2. Use bower to manage front-end asstes
  3. We need to insert dependencies downloaded by bower into HTML layout

This leads us to following problems:

  1. We would like to keep node_modules and bower_components in .gitignore
  2. On local instance we would like to have different assets than in production

Solution: copy assets downloaded from bower and inject them into HTML

Our first problem is we would like to have node_modules and bower_components directories only in developer instance. In our version control we only need configuration files for npm and bower - package.json and bower.json. After cloning the project, developer needs to run following commands to download dependencies:

npm install
bower install

File package.json:

{
  "name": "angular-app",
  "devDependencies": {
    "bower": "^1.7.9",
    "event-stream": "^3.3.2",
    "gulp": "~3.9.0",
    "gulp-angular-filesort": "^1.1.1",
    "gulp-clean-css": "^2.0.9",
    "gulp-concat": "^2.6.0",
    "gulp-if": "^2.0.0",
    "gulp-inject": "^4.1.0",
    "gulp-rename": "^1.2.2",
    "gulp-uglify": "^1.5.1",
    "jasmine-core": "^2.3.4",
    "karma": "~0.13",
    "karma-chrome-launcher": "^0.2",
    "karma-firefox-launcher": "^0.1.6",
    "karma-jasmine": "^0.3.5",
    "karma-junit-reporter": "^0.2.2",
    "main-bower-files": "~2.9.0",
    "yargs": "^3.30.0"
  }
}

File bower.json:

{
  "name": "angular-app",
  "dependencies": {
    "angular": "~1.4.7",
    "angular-route": "~1.4.7",
    "angular-cookies": "~1.4.7",
    "angular-sanitize": "~1.4.7",
    "angular-animate": "~1.4.7",
    "restangular": "~1.5.1",
    "bootstrap": "~3.3.5",
    "angular-bootstrap": "~0.14.3",
    "angular-ui-select": "~0.13.2",
    "angular-bootstrap-show-errors": "~2.3.0",
    "ngtoast": "~1.5.6",
    "angular-ui-tree": "^2.10",
    "angular-loading-bar": "~0.8.0",
    "angular-mocks": "~1.4.0",
    "moment": "~2.10.6"
  }
}

But we don't want to run those npm/bower commands on production server, plus we shouldn't copy node_modules and bower_components through FTP/SFTP (it's often thousands of unnecessary files!)

That's why we should avoid pointing to the bower directory, when referencing external assets because bower_components won't exists on production:

<script src="bower_components/angular/angular.js"></script> // this is incorrect!!

Two gulp plugins are huge help here:

First plugin is using bower.json files located in subdirectories of downloaded components. It looks for a "main" parameter in those files, which describes what are the main files in this component. For example angular component from bower have following parameter in its bower.json file:

{
    ...
    "main": "./angular.js",
    ...
}

Thanks to gulp-main-bower-files plugin we are able to perform some action on those files. For example we can copy them to the app/assets/vendors directory of our app (we can also add this folder to .gitignore). In the gulpfile.js:

var bowerFiles = require('main-bower-files');

var vendorDirectory = './app/assets/vendors';
var bowerStreamJS = gulp.src(bowerFiles('**/*.js'));
var bowerStreamCSS = gulp.src(bowerFiles('**/*.css'));

// copy vendor files from /bower_compontents to /assets/vendors
bowerStreamJS = bowerStreamJS
    .pipe(gulp.dest(vendorDirectory));
bowerStreamCSS = bowerStreamCSS
    .pipe(gulp.dest(vendorDirectory));

Next step is to use gulp-inject plugin in order to automatically inject assets to the HTML layout. First we have to define empty place in the index.html for injection:

<!DOCTYPE html>
<html>
<head>
    ...
    <!-- bower:css -->
    <!-- endinject -->
    ...
</head>
<body>
    ...
    <!-- bower:js -->
    <!-- endinject -->
    ...
</body>
</html>

We add some lines to the gulpfile.js:

// choose source index.html to inject
gulp.src('./app/index.html')
    // send bower bower scripts
    .pipe(
        inject(bowerStreamJS, {relative: true, name: 'bower'})
    )
    // send bower bower css
    .pipe(
        inject(bowerStreamCSS, {relative: true, name: 'bower'})
    // save the file
    .pipe(
        gulp.dest('app')
    );

Running gulp command will copy "main" files from the bower_components to the app/assets/vendors directory and inject them into index.html:

<!DOCTYPE html>
<html>
<head>
    ...
    <!-- bower:css -->
    <link rel="stylesheet" href="assets/vendors/select.css">
    <link rel="stylesheet" href="assets/vendors/ngToast.css">
    <link rel="stylesheet" href="assets/vendors/angular-ui-tree.css">
    <link rel="stylesheet" href="assets/vendors/loading-bar.css">
    <!-- endinject -->
</head>
<body>
    ...
    <!-- bower:js -->
    <script src="assets/vendors/angular.js"></script>
    <script src="assets/vendors/angular-route.js"></script>
    <script src="assets/vendors/angular-cookies.js"></script>
    <script src="assets/vendors/angular-sanitize.js"></script>
    <script src="assets/vendors/angular-animate.js"></script>
    <script src="assets/vendors/lodash.js"></script>
    <script src="assets/vendors/jquery.js"></script>
    <script src="assets/vendors/ui-bootstrap-tpls.js"></script>
    <script src="assets/vendors/select.js"></script>
    <script src="assets/vendors/showErrors.js"></script>
    <script src="assets/vendors/ngToast.js"></script>
    <script src="assets/vendors/angular-ui-tree.js"></script>
    <script src="assets/vendors/loading-bar.js"></script>
    <script src="assets/vendors/angular-mocks.js"></script>
    <script src="assets/vendors/moment.js"></script>
    <script src="assets/vendors/restangular.js"></script>
    <script src="assets/vendors/bootstrap.js"></script>
    <!-- endinject -->
</body>
</html>

We can also add app/assets/vendors directory to the .gitignore. Why? Because we will use separate, uncombined assets only on developer instance. For the production server will need only combined and minified assets - they are located in different directory. For example app/assets/build.

You may ask - why should I copy "main" files from the bower_components to the app/assets/vendors directory and then add both folders to .gitignore? Well it's the matter of my personal preference - I like to keep all my assets in one place so I keep track on what dependencies my app is using. bower_components is a bit obfuscated - different authors use different folder structure so it's often hard to find the file you need during development. Plus having bower assets in app/assets/vendors allows me to verify if the gulp-main-bower-files plugin correctly detected all the files I really need.

Inject custom assets

We would like to also inject our custom JS/CSS file, but we need another inject block for that in our index.html. We have to add to our gulpfile.js:

var appStreamJS = gulp.src(['./app/src/**/*.js'])
    .pipe(angularFilesort());
var appStreamCSS = gulp.src(['./app/src/app.css']);

oraz change:

// choose source index.html to inject
gulp.src('./app/index.html')
    // send bower bower scripts
    .pipe(
        inject(bowerStreamJS, {relative: true, name: 'bower'})
    )
    // send bower bower css
    .pipe(
        inject(bowerStreamCSS, {relative: true, name: 'bower'})
    )
    // send app scripts
    .pipe(
        inject(appStreamJS, {relative: true})
    )
    // send app css
    .pipe(
        inject(appStreamCSS, {relative: true})
    )
    // save the file
    .pipe(
        gulp.dest('app')
    );

We add two inject blocks to the HTML template: <!-- inject:js --> and <!-- inject:css -->. Then after running gulp, our custom assets should appear in the HTML. You probably noticed that I use another gulp plugin here: gulp-angular-filesort, which allows to read AngularJS sources and order them correctly before inject.

But wait! There are incorrect files injected!

Careful readers probably noticed, that Gulp task above injected files like jquery.js or bootstrap.js, which were detected by gulp-main-bower-files, but we don't need them! There are also some files missing, which we actually need like bootstrap.css. In this case gulp-main-bower-files documentation says, that we can override the "main" parameter of dependencies. In our own bower.json file we add following parameters:


{
 ...
  "overrides": {
    "ngtoast": {
      "main": [
        "dist/ngToast.js",
        "dist/ngToast.css",
        "dist/ngToast-animations.css"
      ]
    },
    "jquery": {
      "ignore": true
    },
    "bootstrap": {
      "main": [
        "dist/css/bootstrap.css",
        "fonts/*"
      ]
    }
  }
}

Combining and minifing!

We have already prepared our development instance. We will now concatenate and minify all the assets we injected in the previous step. We are going to save them to the app/assets/build directory, which will be included in the version control. Next, we prepare index.prod.html file which is going to be copy of the development index.html - but it will have minifed, production assets injected instead.

For that we will use following Gulp plugins and NodeJS modules:

The final gulpfile.js is located in the GitHub repository with the complete example from this article: https://github.com/rsobon/angular-gulp-asset-management

After running:

gulp --production

You will have four, combines minified files:

assets/build/app.css // custom app styles
assets/build/app.js // custom app javascript code
assets/build/vendors.css // styles from bower dependencies
assets/build/vendors.js // javascript code from bower dependencies

And the index.prod.html template should look like that:

<!DOCTYPE html>
<html>
<head>
    ...
    <!-- inject:css -->
    <link rel="stylesheet" href="assets/build/app.css">
    <!-- endinject -->

    <!-- bower:css -->
    <link rel="stylesheet" href="assets/build/vendors.css">
    <!-- endinject -->
</head>
<body>
    ...
    <!-- bower:js -->
    <script src="assets/build/vendors.js"></script>
    <!-- endinject -->

    <!-- inject:js -->
    <script src="assets/build/app.js"></script>
    <!-- endinject -->
</body>
</html>

Finishing touch

Now we can serve different HTML file if we are on development and production instance. We can publish the application without fear that production server will get cluttered (also we won't need bower, gulp and npm od production).

Of course, you can configure your project structure differently - for example have index.html hold minified, production assets and maybe keep development file under other name like index.dev.html. You can also use maps for minified assets like CSS and JS (Gulp is supporting map creating too!).

Gulp is very flexible tool, you can adapt it to your own project and workflow!

Repository with the example from the article: https://github.com/rsobon/angular-gulp-asset-management.

We use cookies to ensure you get the best experience on our website. Privacy policy