Removing Sequential Duplicate Words

Alexander often participates in online meetings, and he gets transcripts of the meetings in Word format. Often there are sequential duplicate words in the transcripts, such as "and and" or "it's it's." Alexander would like to delete one of these duplicates, leaving just "and" or "it's." He wonders if there is a way, perhaps with a macro, to look through a document and remove the duplicate words.

There are several different ways you can approach this task. The simplest way is to allow Word to highlight what it thinks are duplicate words. In that way, you can review each instance and decide whether it is appropriate to do the deletion or not. Here's how to make sure Word is configured to do the checking:

1. Display the Word Options dialog box. (In Word 2007 click the Office button and then click Word Options. In Word 2010 and later versions, display the File tab of the ribbon and then click Options.)
2. Click Proofing at the left side of the screen. (See Figure 1.)



Figure 1. The Proofing options of the Word Options dialog box.

3. Make sure the Flag Repeated Words check box is selected.
4. Click OK.

Now Word will add a red underline to the second word in a sequential series of repeated words.

If you have a lot of duplicates in your document, you may want to bypass the manual review (and attendant manual editing) and, instead, let a macro do the deletions for you. Provided you have the proofing option set as just discussed, this is a relatively quick thing to do by stepping through the SpellingErrors collection, in the following manner:

Sub RemoveDuplicateWords()
    Dim Errs As ProofreadingErrors
    Dim ErrWrd As Range
    Dim sRange As Range
    Dim prevWrd As Range
    Dim j As Long
    Dim k As Long

    k = 0
    Set Errs = ActiveDocument.SpellingErrors
    For Each ErrWrd In Errs
        Set sRange = ErrWrd.Sentences(1)
        sRange.End = ErrWrd.End
        j = sRange.Words.Count
        If j > 1 Then
            Set prevWrd = sRange.Words(j - 1)
            If Trim(LCase(prevWrd.Text)) = Trim(LCase(ErrWrd.Text)) Then
                k = k + 1
                ErrWrd.MoveStart unit:=wdCharacter, Count:=-1
                ErrWrd.Text = ""
            End If
        End If
    Next ErrWrd
    MsgBox k & " duplicate words deleted"
End Sub

The macro grabs the text of each spelling error (in the ErrWrd range) and then examines the word just before that error (in the prevWrd range). If they are the same, then the second word is deleted, along with the space just before it.

Obviously, in any macro such as this you'll want to make sure you have a copy of the document before you run it. That way you'll be able to rely on the copy if the macro deletes more text than you would like.

You could, as well, use a macro to step through each word in a document (the Words collection). That would allow you to perform an even wider range of processing with the duplicates. Here's an example:

Sub FixWordDups()
    Dim iDups As Integer
    Dim lWdCnt As Long
    Dim lWdPtr As Long
    Dim a As String
    Dim b As String
    Dim sMsg As String
    Dim sTemp As String

    lWdCnt = ActiveDocument.Words.Count
    iDups = 0

    sMsg = "1. Highlight" & vbCr & "2. Make bold" & vbCr
    sMsg = sMsg & "3. Make red" & vbCr & "4. Delete"

    sTemp = InputBox(sMsg, "Enter the desired action")
    iAction = Cint(Trim(sTemp))
    If iAction <1 Or iAction > 4 Then
        Beep
        Exit Sub
    End If

    With ActiveDocument
        For lWdPtr = 2 To lWdCnt
            a = LCase(.Words(lWdPtr - 1))
            b = LCase(.Words(lWdPtr))

            If a = b Then       ' Words are the same
                iDups = iDups + 1
                Select Case iAction
                    Case 1      ' Highlight
                        .Words(lWdPtr).HighlightColorIndex = wdTurquoise
                    Case 2      ' Bold
                        .Words(lWdPtr).Bold = True
                    Case 3      ' Red
                        .Words(lWdPtr).Font.ColorIndex = wdRed
                    Case 4      ' Delete
                        .Words(lWdPtr).Delete
                        lWdCnt = lWdCnt - 1
                        lWdPtr = lWdPtr - 1
                End Select
            End If
        Next lWdPtr
    End With
    Application.ScreenRefresh
    MsgBox iDups & " duplicated words were found and processed."
End Sub

This macro asks the user how duplicates should be handled, providing four options. The words are stepped through and, if a duplicate is detected, the desired action is completed. When done, the macro displays an indication of the number of duplicates that were detected and processed.

If you prefer to not use a macro for some reason, you can also try using Find and Replace to globally remove any duplicate words. Follow these steps:

1. Press Ctrl+H. Word displays the Replace tab of the Find and Replace dialog box.
2. Click the More button if it is available. Word expands the Find and Replace dialog box. (See Figure 2.)



Figure 2. The Replace tab of the Find and Replace dialog box.

3. Enter the following in the Find What box:

        (<[A-Za-z']{1,}>) \1([ .,;:\!\?^13\]\)])

4. Enter the following in the Replace With box: \1\2
5. Make sure the Use Wildcards check box is selected.
6. Click on Replace All.
7. Close the Find and Replace dialog box.

The key to getting this approach to work correctly is the Find What pattern entered in step 3. This pattern looks for any duplicated words that are separated by a single space. (If there is punctuation between the words, they won't match the pattern.)

The drawback to this approach is that it will only find words consisting of letters. If you believe that your document might have double words that contain digits, or double words that are hyphenated, then this approach won't work. Plus, it only matches double words where the case of the words match. So, it would match "Holding Holding," but not "Holding holding." In any of these cases, you'll want to start with one of the macros previously described and modify them to fit your specific needs.

WordTips is your source for cost-effective Microsoft Word training. (Microsoft Word is the most popular word processing software in the world.) This tip (895) applies to Microsoft Word 2007, 2010, 2013, 2016, 2019, Word in Microsoft 365, and 2021.