How to remove duplicates from an array in JavaScript

In this post we will be covering how to remove duplicates from an array in JavaScript with as little technical jargon as possible so you will have everything you need right here without having to look any further!

In JavaScript, it is pretty common to find yourself needing to remove duplicates from an array.

If the duplicates can be avoided in the first place then that will always be the best course of action, but sometimes it can’t be helped and this is why we need to be able to remove duplicates from an array in JavaScript.

There are a few different ways in which we can do this, some easier than others, so let’s get started and look into how to remove duplicates from an array in JavaScript.

How to remove duplicates from an array in JavaScript

The easiest way to remove duplicates from an array in JavaScript is to convert the array into a set and then convert it back into an array.

A set in JavaScript let’s you store a collection of unique values and by converting your array into a set it will automatically remove any duplicates for you.

A set can be created using the Set constructor in JavaScript.

Here is an example of how you can remove duplicates from an array in JavaScript by using a Set.

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
new Set(anArrayWithDuplicates) // Set(4) {4, 5, 6, 7}

As you can see from the above example by using a set we have removed all duplicates from the array.

This might be more than enough in most instances, but sometimes you might need to convert the set back into an array depending on what you need to do.

To convert the set back into an array all we need to do is use the spread syntax on the set into a new array.

Here is an example of how to do this:

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7] // [4, 5, 6, 4, 5, 7]
const deduplicatedArray = [...new Set(anArrayWithDuplicates)] // [4, 5, 6, 7]

As you can see we have now removed all duplicates from the array in JavaScript using the set object.

This method of deduplicating an array can be used in most modern browsers since ES6. It is important to remember though that this won’t work in Internet explorer because it does not support the Set object.

How to remove duplicates from an array in JavaScript using a for loop

To remove duplicates from an array in JavaScript using a for loop, it is a little different to the above, and there is no single way to do it.

To remove duplicates from an array using a for loop will require an algorithm. The best case we can look for is a time space complexity of O(n) because no matter what we will always need to check every item in the array to find any duplicated counterparts.

If you aren’t sure what big o notation means that is okay, all O(n) means is that we are performing at least one, non-nested loop which is what the n means.

A time space complexity of O(n) is pretty good and will be fine to use even with larger arrays.

The best time space complexity you can have for any algorithm is O(1) which means constant time.

Where we are needing to loop over an array though it can’t be constant.

Considering this, we know we need to look through each item in the array at least once and then find if any of those items have been duplicated.

Here is one example of remove duplicates from an array in JavaScript using a for loop:

const dedupeArray = someArray => {
  const history = {}
  const newDeduplicatedArray = []

  for (let i = 0; i < someArray.length; i += 1) {
    if (!history?.[someArray[i]]) {
      newDeduplicatedArray.push(someArray[i])
      history[someArray[i]] = true
    }
  }

  return newDeduplicatedArray
}

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
dedupeArray(anArrayWithDuplicates) // [4, 5, 6, 7]

In this above example, we are using an object to keep track of values that we have already seen in the array, if we then see that value in the history object then we know it is duplicated so we don’t add it to the new array twice which then gives us an array with all the duplicates removed.

The algorithm above is not necessarily the best and can probably be improved, but it does give us a space time complexity of O(n) which is the best we can hope for here and it gets the job done because we only perform one loop that has not been nested.

Another way to remove duplicates from an array in JavaScript using a for loop would be to first sort the array, and then check the index in front of the one you are looking at. This way you will easily spot any duplicate values.

It is common practice when dealing with algorithms around arrays to first sort your array because sorting algorithms can be pretty fast and as long as they have a better time complexity to your solution then it can a good idea to sort the array first.

Here is an example of sorting your array first to remove duplicates in an array in JavaScript:

const dedupeArray = someArray => {
  const sortedArray = [...someArray].sort((n, n1) => {
    if (n > n1) return 1
    if (n < n1) return -1
    return 0
  })

  const newDeduplicatedArray = []

  for (let i = 0; i < sortedArray.length; i += 1) {
    if (sortedArray?.[i + 1] !== sortedArray[i]) {
      newDeduplicatedArray.push(sortedArray[i])
    }
  }

  return newDeduplicatedArray
}

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
dedupeArray(anArrayWithDuplicates) // [4, 5, 6, 7]

How to remove duplicates from an array in JavaScript using reduce

To remove duplicates from an array in JavaScript using reduce we can implement something similar to what we used for the for loop in the above example.

You could use a history object, or sort the array first, but to remove duplicates purley inside of the reduce function makes it a little trickier.

You could still store and pass the values as part of the accumulator like so:

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
anArrayWithDuplicates.reduce(
  (acc, arrayItem) => {
    const history = { ...acc.history }
    const deduplicated = [...acc.deduplicated]

    if (!history?.[arrayItem]) {
      deduplicated.push(arrayItem)
      history[arrayItem] = true
    }

    return { history, deduplicated }
  },
  { deduplicated: [], history }
)

Or you could instead look in the array you are creating to see if the value has already been added:

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
anArrayWithDuplicates.reduce((acc, arrayItem) => {
  const deduplicated = [...acc]

  if (!deduplicated.includes(arrayItem)) {
    deduplicated.push(arrayItem)
  }

  return deduplicated
}, [])

As you can see both of these examples work to deduplicate and remove duplicates from an array in JavaScript using reduce, but in general using reduce is not the most ideal way to remove duplicates from an array because it can become a little confusing, instead using a forEach or a for loop would be better suited if you cannot deduplicate them using a set like in our first example.

How to find duplicates in an array in JavaScript

Finding duplicates in an array can be done so in a very similar way in our for loop example with a small change.

const anArrayWithDuplicates = [4, 5, 6, 4, 5, 7]
const history = {}
const duplicates = []

for (let i = 0; i < anArrayWithDuplicates.length; i += 1) {
  if (history?.[anArrayWithDuplicates[i]]) {
    duplicates.push(anArrayWithDuplicates[i])
  }
  history[anArrayWithDuplicates[i]] = true
}

console.log(duplicates) // [4, 5]

With that small modification our for loop will now find all duplicate values in the array, this could also be done using the sort method as well.

Summary

There we have how to remove duplicates from an array in JavaScript, if you want more like this be sure to check out some of my other posts!