Swift JSONDecode decoding arrays fails if single element decoding fails

While using Swift4 and Codable protocols I got the following problem - it looks like there is no way to allow JSONDecoder to skip elements in an array. For example, I have the following JSON:

[
    {
        "name": "Banana",
        "points": 200,
        "description": "A banana grown in Ecuador."
    },
    {
        "name": "Orange"
    }
]

And a Codable struct:

struct GroceryProduct: Codable {
    var name: String
    var points: Int
    var description: String?
}

When decoding this json

let decoder = JSONDecoder()
let products = try decoder.decode([GroceryProduct].self, from: json)

Resulting products is empty. Which is to be expected, due to the fact that the second object in JSON has no "points" key, while points is not optional in GroceryProduct struct.

Question is how can I allow JSONDecoder to "skip" invalid object?

One option is to use a wrapper type that attempts to decode a given value; storing nil if unsuccessful:

struct FailableDecodable<Base : Decodable> : Decodable {

    let base: Base?

    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        self.base = try? container.decode(Base.self)
    }
}

We can then decode an array of these, with your GroceryProduct filling in the Base placeholder:

import Foundation

let json = """
[
    {
        "name": "Banana",
        "points": 200,
        "description": "A banana grown in Ecuador."
    },
    {
        "name": "Orange"
    }
]
""".data(using: .utf8)!


struct GroceryProduct : Codable {
    var name: String
    var points: Int
    var description: String?
}

let products = try JSONDecoder()
    .decode([FailableDecodable<GroceryProduct>].self, from: json)
    .compactMap { $0.base } // .flatMap in Swift 4.0

print(products)

// [
//    GroceryProduct(
//      name: "Banana", points: 200,
//      description: Optional("A banana grown in Ecuador.")
//    )
// ]

We're then using .compactMap { $0.base } to filter out nil elements (those that threw an error on decoding).

This will create an intermediate array of [FailableDecodable<GroceryProduct>], which shouldn't be an issue; however if you wish to avoid it, you could always create another wrapper type that decodes and unwraps each element from an unkeyed container:

struct FailableCodableArray<Element : Codable> : Codable {

    var elements: [Element]

    init(from decoder: Decoder) throws {

        var container = try decoder.unkeyedContainer()

        var elements = [Element]()
        if let count = container.count {
            elements.reserveCapacity(count)
        }

        while !container.isAtEnd {
            if let element = try container
                .decode(FailableDecodable<Element>.self).base {

                elements.append(element)
            }
        }

        self.elements = elements
    }

    func encode(to encoder: Encoder) throws {
        var container = encoder.singleValueContainer()
        try container.encode(elements)
    }
}

You would then decode as:

let products = try JSONDecoder()
    .decode(FailableCodableArray<GroceryProduct>.self, from: json)
    .elements

print(products)

// [
//    GroceryProduct(
//      name: "Banana", points: 200,
//      description: Optional("A banana grown in Ecuador.")
//    )
// ]

I would create a new type Throwable, which can wrap any type conforming to Decodable:

enum Throwable<T: Decodable>: Decodable {
    case success(T)
    case failure(Error)

    init(from decoder: Decoder) throws {
        do {
            let decoded = try T(from: decoder)
            self = .success(decoded)
        } catch let error {
            self = .failure(error)
        }
    }
}

For decoding an array of GroceryProduct (or any other Collection):

let decoder = JSONDecoder()
let throwables = try decoder.decode([Throwable<GroceryProduct>].self, from: json)
let products = throwables.compactMap { $0.value }

where value is a computed property introduced in an extension on Throwable:

extension Throwable {
    var value: T? {
        switch self {
        case .failure(_):
            return nil
        case .success(let value):
            return value
        }
    }
}

I would opt for using a enum wrapper type (over a Struct) because it may be useful to keep track of the errors that are thrown as well as their indices.

Swift 5

For Swift 5 Consider using the Result enum e.g.

struct Throwable<T: Decodable>: Decodable {
    let result: Result<T, Error>

    init(from decoder: Decoder) throws {
        result = Result(catching: { try T(from: decoder) })
    }
}

To unwrap the decoded value use the get() method on the result property:

let products = throwables.compactMap { try? $0.result.get() }

The problem is that when iterating over a container, the container.currentIndex isn’t incremented so you can try to decode again with a different type.

Because the currentIndex is read only, a solution is to increment it yourself successfully decoding a dummy. I took @Hamish solution, and wrote a wrapper with a custom init.

This problem is a current Swift bug: https://bugs.swift.org/browse/SR-5953

The solution posted here is a workaround in one of the comments. I like this option because I’m parsing a bunch of models the same way on a network client, and I wanted the solution to be local to one of the objects. That is, I still want the others to be discarded.

I explain better in my github https://github.com/phynet/Lossy-array-decode-swift4

import Foundation

    let json = """
    [
        {
            "name": "Banana",
            "points": 200,
            "description": "A banana grown in Ecuador."
        },
        {
            "name": "Orange"
        }
    ]
    """.data(using: .utf8)!

    private struct DummyCodable: Codable {}

    struct Groceries: Codable 
    {
        var groceries: [GroceryProduct]

        init(from decoder: Decoder) throws {
            var groceries = [GroceryProduct]()
            var container = try decoder.unkeyedContainer()
            while !container.isAtEnd {
                if let route = try? container.decode(GroceryProduct.self) {
                    groceries.append(route)
                } else {
                    _ = try? container.decode(DummyCodable.self) // <-- TRICK
                }
            }
            self.groceries = groceries
        }
    }

    struct GroceryProduct: Codable {
        var name: String
        var points: Int
        var description: String?
    }

    let products = try JSONDecoder().decode(Groceries.self, from: json)

    print(products)

There are two options:

Declare all members of the struct as optional whose keys can be missing

struct GroceryProduct: Codable {
    var name: String
    var points : Int?
    var description: String?
}

Write a custom initializer to assign default values in the nil case.

struct GroceryProduct: Codable {
    var name: String
    var points : Int
    var description: String

    init(from decoder: Decoder) throws {
        let values = try decoder.container(keyedBy: CodingKeys.self)
        name = try values.decode(String.self, forKey: .name)
        points = try values.decodeIfPresent(Int.self, forKey: .points) ?? 0
        description = try values.decodeIfPresent(String.self, forKey: .description) ?? ""
    }
}

Swift JSONDecode decoding arrays fails if single element decoding fails

Swift 5

Related

Recent Posts