Regex for EDT with AI

Question
1

I would like to test the artificial intelligence for creating regexes compatible with Easydatatransform. What are the constraints or standards that I need to indicate to the AI ​​in order to create the right regex?

2

We haven’t tried to use an AI to create a regular expression yet, so we can’t really comment. Perhaps someone else can?

1 Like
3

maybe this could help…
As Admin had stated: The Easy Data Transform regex is based on Perl regex.

1 Like
4

You can use chatgpt to generate regex or even javascript your require @ANTONIA . Here’s a prompt I use with chatgpt which generates good enough javascripts 99% of times. Maybe a prompt can be created for regex too

I want to make javascript script but I want you to follow these guidelines

Once you understand please give yes/no so I can proceed

Creating JavaScript for Easy Data Transform (EDT) involves some best practices and considerations to ensure efficient and error-free data transformations. Here are some key points:

Key Features and Best Practices:

  1. Variable Handling:

    • Use $(column_number) to reference column values directly.
    • Always handle potential null or undefined values to prevent errors.

  2. String Operations:

    • Use .trim() to remove any extra whitespace from strings.
    • Use .split(delimiter) to divide strings based on a specific character.

  3. Date Handling:

    • Ensure date strings are in a format that JavaScript’s Date object can parse (e.g., ISO format yyyy-mm-dd).
    • Always validate date strings before parsing to avoid Invalid Date errors.

  4. Conditional Logic:

    • Utilize if-else statements to handle different scenarios, such as checking for delimiters or specific value conditions.

  5. Error Handling:

    • Include checks and fallback values to handle unexpected data gracefully.
    • Use try-catch blocks if performing complex operations that may throw exceptions.

  6. Code Readability:

    • Write clear and concise code with comments explaining key sections.
    • Use meaningful variable names for clarity.

Things to Avoid:

  1. Complex Calculations:

    • Avoid overly complex calculations within the JavaScript transform. Use built-in transforms where possible for efficiency.

  2. Unnecessary Global Variables:

    • Avoid using global variables. Keep variables scoped within the function to prevent unintended side effects.

  3. Heavy Computations:

    • EDT’s JavaScript transform is not optimized for heavy computations. Offload such tasks to dedicated scripts or applications if needed.

  4. Inconsistent Date Formats:

    • Ensure date formats are consistent across your dataset to avoid parsing issues.

  5. Ignoring Edge Cases:

    • Always consider edge cases (e.g., empty strings, incorrect formats) to make your script robust.

Nitpicks and Extra Tips:

  1. Default Values:

    • Provide default values in case of missing or invalid data. For example:
      var dateString = $(18) || '';

  2. Regular Expressions:

    • Use regular expressions for advanced string manipulations, but ensure they are efficient and well-tested.

  3. Avoid Hardcoding Column Numbers:

    • If possible, use dynamic column referencing or document the column numbers to make the script easier to update.

  4. Efficiency:

    • Optimize loops and conditionals to run efficiently, especially on large datasets.

  5. Testing:

    • Test the script with a subset of your data to ensure it works as expected before applying it to the entire dataset.

Example: Robust Script for Finding Earlier Date:

// Retrieve the value from column 18 and handle missing data
var dateString = $(18) || '';

// Function to safely parse dates
function safeParseDate(dateStr) {
  var date = new Date(dateStr.trim());
  return isNaN(date) ? null : date;
}

// Check if the column contains the delimiter '|'
if (dateString.includes('|')) {
  // Split the dates
  var dates = dateString.split('|');
  
  // Parse dates using the safe parse function
  var date1 = safeParseDate(dates[0]);
  var date2 = safeParseDate(dates[1]);
  
  // Return the earlier date if both are valid, else handle invalid dates
  if (date1 && date2) {
    return date1 < date2 ? dates[0].trim() : dates[1].trim();
  } else if (date1) {
    return dates[0].trim();
  } else if (date2) {
    return dates[1].trim();
  } else {
    return ''; // or handle as needed
  }
} else {
  // Return the original value if no delimiter found or handle as needed
  return dateString;
}

By following these best practices and considerations, you can create efficient and reliable JavaScript transforms in Easy Data Transform.

2 Likes
5

Thanks a lot. i did not try javascript but only regex delimiter but I think that I misunderstood its function, because i tried alot without success to split a column in three columns using regex to detect the first two spaces in a full name.

1 Like