Karate API Testing- Remove duplicate values from the response and compare it with the new response

I have a scenario where in response I receive duplicate values.

And def abcName =  $ListDataSet_Response.rowData[*].4

And print abcName

 [
 "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA", 
  "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
  "BP Mrts Block NCA MS"

  ]

Now I have a feature where with distinct values query I can get response without duplicate values.After performing that operation .New response is:

  [
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
   "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
  ]

Now I need to validate this response with the earlier one keeping in mind that if I remove duplicate values from the first response, it will be my new response (received from the distinct values quesry).

All these values are run time and dynamic.

My Approach till now:

First:

1. Storing abcName in a set(Java). It will remove the duplicate.

2. Now compare it with the new response.

   This is not feasible as both are of different type.

   Second:

3. Storing abcName in a set(Java)

4. store second response in another set (This will go against the purpose as it should be in a set format)

   Third:

5. Storing abcName in a set(Java)

6. convert this Set to list : This is failing
7. store second response in a list

8. Now compare these two list

   Is there any way in karate where without using Java we can do this activity?

* def response = 
"""
[
 "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA", 
  "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
  "BP Mrts Block NCA MS"
]
"""
* json response = new java.util.HashSet(response)
* def expected =
"""
[
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
   "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
]
"""
* match response contains only expected

Also see: https://stackoverflow.com/a/68065532/143475

For anyone who comes across this, in 1.0+ the above answer no longer works - you now need to use karate.toJava() before converting it to a HashSet, i.e.:

* def response = 
"""
[
 "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA",
  "BP CS Sht NCA", 
  "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
  "BP Mrts Block NCA MS"
]
"""
* json response = new java.util.HashSet(karate.toJava(response))
* def expected =
"""
[
  "BP Part Sht NCA MS",
  "Bay Pond USB, Inc MS",
  "BP USB III Inc MS",
  "BP CS Sht NCA",
   "BP USB IV, Inc MS",
  "BP Mrts Block NCA MS",
]
"""
* match response contains only expected