diff options
| author |   Spartan322 <Megacake1234@gmail.com> | 2024-05-09 16:06:02 +0200 |
|---|---|---|
| committer | Spartan322 <Megacake1234@gmail.com> | 2024-06-18 01:31:12 +0200 |
| commit | b0c3ba3f91926b0c95625bdbf4aab69269130b13 (patch) | |
| tree | f15ebc47d6bf370031af28e4bb4814ae30ef46e1 /src/openvic-dataloader/csv/CsvParseState.hpp | |
| parent | 7b521d6023113372cf6b02e562828273c4040f0e (diff) | |
Add runtime encoding detection and conversionfix/char-detection
Win-1251/1252 detection is a reduced C++ version of https://github.com/hsivonen/chardetng
Add manually-specified encoding fallback
Add default system encoding fallback
Add error recovery to v2script
Add unknown encoding detection warning
Remove csv::Parser templating
Fix lua files dropping data
Update lexy to foonathan/lexy@1e5d99fa3826b1c3c8628d3a11117fb4fb4cc0d0
Remove exclusive reliance on lexy::default_encoding for v2script
Move internal concepts to src/openvic-detail/InternalConcepts.hpp
Move contents of DetectUtf8.hpp to src/detail/Detect.hpp
Move openvic-dataloader/AbstractSyntaxTree.hpp to src
Move DiagnosticLogger.hpp to src
Move File.hpp to src
Move openvic-dataloader/detail/utlity files to openvic-dataloader/detail
Add ovdl::utility::type_concat
Add ovdl::utility::type_prepend
Add ovdl::utility::is_instance_of
Overhaul parse error messages
Diffstat (limited to 'src/openvic-dataloader/csv/CsvParseState.hpp')
| -rw-r--r-- | src/openvic-dataloader/csv/CsvParseState.hpp | 26 |
1 files changed, 7 insertions, 19 deletions
diff --git a/src/openvic-dataloader/csv/CsvParseState.hpp b/src/openvic-dataloader/csv/CsvParseState.hpp index 2390453..ee60c34 100644 --- a/src/openvic-dataloader/csv/CsvParseState.hpp +++ b/src/openvic-dataloader/csv/CsvParseState.hpp @@ -1,28 +1,16 @@ #pragma once -#include <openvic-dataloader/File.hpp> -#include <openvic-dataloader/ParseState.hpp> #include <openvic-dataloader/csv/LineObject.hpp> #include <openvic-dataloader/csv/Parser.hpp> #include <lexy/encoding.hpp> -template<ovdl::csv::EncodingType> -struct LexyEncodingFrom { -}; +#include "File.hpp" +#include "ParseState.hpp" +#include "detail/InternalConcepts.hpp" -template<> -struct LexyEncodingFrom<ovdl::csv::EncodingType::Windows1252> { - using encoding = lexy::default_encoding; -}; +namespace ovdl::csv { + using CsvParseState = ovdl::FileParseState<ovdl::BasicFile<std::vector<ovdl::csv::LineObject>>>; -template<> -struct LexyEncodingFrom<ovdl::csv::EncodingType::Utf8> { - using encoding = lexy::utf8_char_encoding; -}; - -template<ovdl::csv::EncodingType Encoding> -using CsvFile = ovdl::BasicFile<typename LexyEncodingFrom<Encoding>::encoding, std::vector<ovdl::csv::LineObject>>; - -template<ovdl::csv::EncodingType Encoding> -using CsvParseState = ovdl::FileParseState<CsvFile<Encoding>>;
\ No newline at end of file + static_assert(detail::IsFileParseState<CsvParseState>, "CsvParseState failed IsFileParseState concept"); +}
\ No newline at end of file |