Removing Sequential Duplicate Words

Written by Allen Wyatt (last updated November 11, 2023)
This tip applies to Word 2007, 2010, 2013, 2016, 2019, Word in Microsoft 365, and 2021


2

Alexander often participates in online meetings, and he gets transcripts of the meetings in Word format. Often there are sequential duplicate words in the transcripts, such as "and and" or "it's it's." Alexander would like to delete one of these duplicates, leaving just "and" or "it's." He wonders if there is a way, perhaps with a macro, to look through a document and remove the duplicate words.

There are several different ways you can approach this task. The simplest way is to allow Word to highlight what it thinks are duplicate words. In that way, you can review each instance and decide whether it is appropriate to do the deletion or not. Here's how to make sure Word is configured to do the checking:

  1. Display the Word Options dialog box. (In Word 2007 click the Office button and then click Word Options. In Word 2010 and later versions, display the File tab of the ribbon and then click Options.)
  2. Click Proofing at the left side of the screen. (See Figure 1.)
  3. Figure 1. The Proofing options of the Word Options dialog box.

  4. Make sure the Flag Repeated Words check box is selected.
  5. Click OK.

Now Word will add a red underline to the second word in a sequential series of repeated words.

If you have a lot of duplicates in your document, you may want to bypass the manual review (and attendant manual editing) and, instead, let a macro do the deletions for you. Provided you have the proofing option set as just discussed, this is a relatively quick thing to do by stepping through the SpellingErrors collection, in the following manner:

Sub RemoveDuplicateWords()
    Dim Errs As ProofreadingErrors
    Dim ErrWrd As Range
    Dim sRange As Range
    Dim prevWrd As Range
    Dim j As Long
    Dim k As Long

    k = 0
    Set Errs = ActiveDocument.SpellingErrors
    For Each ErrWrd In Errs
        Set sRange = ErrWrd.Sentences(1)
        sRange.End = ErrWrd.End
        j = sRange.Words.Count
        If j > 1 Then
            Set prevWrd = sRange.Words(j - 1)
            If Trim(LCase(prevWrd.Text)) = Trim(LCase(ErrWrd.Text)) Then
                k = k + 1
                ErrWrd.MoveStart unit:=wdCharacter, Count:=-1
                ErrWrd.Text = ""
            End If
        End If
    Next ErrWrd
    MsgBox k & " duplicate words deleted"
End Sub

The macro grabs the text of each spelling error (in the ErrWrd range) and then examines the word just before that error (in the prevWrd range). If they are the same, then the second word is deleted, along with the space just before it.

Obviously, in any macro such as this you'll want to make sure you have a copy of the document before you run it. That way you'll be able to rely on the copy if the macro deletes more text than you would like.

You could, as well, use a macro to step through each word in a document (the Words collection). That would allow you to perform an even wider range of processing with the duplicates. Here's an example:

Sub FixWordDups()
    Dim iDups As Integer
    Dim lWdCnt As Long
    Dim lWdPtr As Long
    Dim a As String
    Dim b As String
    Dim sMsg As String
    Dim sTemp As String

    lWdCnt = ActiveDocument.Words.Count
    iDups = 0

    sMsg = "1. Highlight" & vbCr & "2. Make bold" & vbCr
    sMsg = sMsg & "3. Make red" & vbCr & "4. Delete"

    sTemp = InputBox(sMsg, "Enter the desired action")
    iAction = Cint(Trim(sTemp))
    If iAction <1 Or iAction > 4 Then
        Beep
        Exit Sub
    End If

    With ActiveDocument
        For lWdPtr = 2 To lWdCnt
            a = LCase(.Words(lWdPtr - 1))
            b = LCase(.Words(lWdPtr))

            If a = b Then       ' Words are the same
                iDups = iDups + 1
                Select Case iAction
                    Case 1      ' Highlight
                        .Words(lWdPtr).HighlightColorIndex = wdTurquoise
                    Case 2      ' Bold
                        .Words(lWdPtr).Bold = True
                    Case 3      ' Red
                        .Words(lWdPtr).Font.ColorIndex = wdRed
                    Case 4      ' Delete
                        .Words(lWdPtr).Delete
                        lWdCnt = lWdCnt - 1
                        lWdPtr = lWdPtr - 1
                End Select
            End If
        Next lWdPtr
    End With
    Application.ScreenRefresh
    MsgBox iDups & " duplicated words were found and processed."
End Sub

This macro asks the user how duplicates should be handled, providing four options. The words are stepped through and, if a duplicate is detected, the desired action is completed. When done, the macro displays an indication of the number of duplicates that were detected and processed.

If you prefer to not use a macro for some reason, you can also try using Find and Replace to globally remove any duplicate words. Follow these steps:

  1. Press Ctrl+H. Word displays the Replace tab of the Find and Replace dialog box.
  2. Click the More button if it is available. Word expands the Find and Replace dialog box. (See Figure 2.)
  3. Figure 2. The Replace tab of the Find and Replace dialog box.

  4. Enter the following in the Find What box:
  5.         (<[A-Za-z']{1,}>) \1([ .,;:\!\?^13\]\)])
    
  6. Enter the following in the Replace With box: \1\2
  7. Make sure the Use Wildcards check box is selected.
  8. Click on Replace All.
  9. Close the Find and Replace dialog box.

The key to getting this approach to work correctly is the Find What pattern entered in step 3. This pattern looks for any duplicated words that are separated by a single space. (If there is punctuation between the words, they won't match the pattern.)

The drawback to this approach is that it will only find words consisting of letters. If you believe that your document might have double words that contain digits, or double words that are hyphenated, then this approach won't work. Plus, it only matches double words where the case of the words match. So, it would match "Holding Holding," but not "Holding holding." In any of these cases, you'll want to start with one of the macros previously described and modify them to fit your specific needs.

Note:

If you would like to know how to use the macros described on this page (or on any other page on the WordTips sites), I've prepared a special page that includes helpful information. Click here to open that special page in a new browser tab.

WordTips is your source for cost-effective Microsoft Word training. (Microsoft Word is the most popular word processing software in the world.) This tip (895) applies to Microsoft Word 2007, 2010, 2013, 2016, 2019, Word in Microsoft 365, and 2021.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

More than Three Conditional Formats

Conditional formatting is a great feature for making the data in your worksheets more understandable and usable. What if ...

Discover More

Breaking Up Variable-Length Part Numbers

Part numbers can often be long, made up of other component elements. Breaking up part numbers into individual components ...

Discover More

Quickly Changing Tab Alignment

Need to change the alignment of some tabs in a multitude of paragraphs? How you approach the problem depends on whether ...

Discover More

Comprehensive VBA Guide Visual Basic for Applications (VBA) is the language used for writing macros in all Office programs. This complete guide shows both professionals and novices how to master VBA in order to customize the entire Office suite for their needs. Check out Mastering VBA for Office 2010 today!

More WordTips (ribbon)

When to Hyphenate Your Document

Want to add a finishing touch to your printed document? Hyphenating the words in the document can make your text read ...

Discover More

Working with Master and Subdocuments

Word has long had the capability of establishing relationships between documents by designating some as master documents ...

Discover More

Jumping to the End of the Document in Error

If you inadvertently move to the end of the document, you might be wondering how to get your insertion point back to ...

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in WordTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments

If you would like to add an image to your comment (not an avatar, but an image to help in making the point of your comment), include the characters [{fig}] (all 7 characters, in the sequence shown) in your comment text. You’ll be prompted to upload your image when you submit the comment. Maximum image size is 6Mpixels. Images larger than 600px wide or 1000px tall will be reduced. Up to three images may be included in a comment. All images are subject to review. Commenting privileges may be curtailed if inappropriate images are posted.

What is 2 + 5?

2023-11-11 10:08:03

Ron S

When you are doing document proofreading or editing someone else's document you may need to do repetitive actions. A cue you want to use a macro.

There is a free ebook you can download written by a professional book editor that contains a collection of macros he uses. It introduces a powerful Find/Replace/Edit macro tool he calls "FREdit"



Macros for Editors (Download)
http://archivepub.co.uk/book.html
by Paul Beverly.
This is a free book, which you can download (in zip format) (version: 20 Jan 2022). It contains over 800 macros that will help with a range of different tasks around writing and editing using Microsoft Word.

Table of Contents
General introduction
Introduction to macros
Downloading and running macros
Favourite tools of editors
Favourite tools of proofreaders
Proofreading a book – a possible workflow
Book editing – a possible workflow
Tools for different aspects of editing
Macro Menu – complete macro tool list
Textual analysis
Main pre-editing tool – FRedit
FRedit Manual
Pre-editing tools
Editing – text change
Editing – information
Editing – highlighting
Editing – navigation
Editing – comment handling
Editing – track changes
Other tools
Appendix 1 – Codes for F&R
Appendix 2 – Codes for non-wildcard F&R
Appendix 3 – Codes for wildcard F&R
Appendix 4 – Some useful wildcard expressions
Appendix 5 – ASCII codes
Appendix 6 – Useful unicode numbers
Appendix 7 – Sample stylesheet
Appendix 8 – Sample FRedit list
Appendix 9 – Word 365 options
Appendix 10 – Word 365 menu items
Appendix 11 – Word 2010 options
Appendix 12 – Backing up the Normal Template
Appendix 13 – Word Macro Techniques
Appendix 14 – Video list
Appendix 15 – Macros on video
Appendix 16 – FRedit library
Changes log


2023-11-11 06:08:45

Ken Endacott

The macros RemoveDuplicateWords and FixWordDups will remove duplicate words that are separated by spaces but not by other delimiters such a period or semicolon. This is usually what you want so that a word ending a sentence can be repeated at the start of the next sentence.

FixWordDups requires the same number of spaces following each word but RemoveDuplicateWords will remove the second word even if there are differing numbers of spaces after each word, for example if the second word is at the end of a sentence.

To make FixWordDups do the same, change the line:
If a = b Then
to,
If Trim(a) = Trim(b) Then


This Site

Got a version of Word that uses the ribbon interface (Word 2007 or later)? This site is for you! If you use an earlier version of Word, visit our WordTips site focusing on the menu interface.

Videos
Subscribe

FREE SERVICE: Get tips like this every week in WordTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.